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ABSTRACT 


Syndromic  Surveillance  and  Outbreak  Detection  Using  Automated  Microbiologic 

Laboratory  Test  Order  Data 

Cara  Hendricks  Olsen 
Doctor  of  Public  Health  Degree,  2007 

Thesis  directed  by  David  F.  Cruess,  PhD,  Department  of  Preventive  Medicine  and 

Biometrics 

Background:  Syndromic  surveillance  systems  monitor  one  or  more  electronic  data 
sources  in  real  time  to  assist  in  early  detection  of  unusual  health  events.  To  detect  such 
events  at  military  treatment  facilities  (MTFs),  the  Department  of  Defense  Electronic 
Surveillance  System  for  the  Early  Notification  of  Community-based  Epidemics  (DoD- 
ESSENCE)  conducts  daily  surveillance  on  outpatient  visit  diagnosis  and  pharmacy  data. 
Combining  data  from  multiple  sources  may  improve  the  ability  of  syndromic  surveillance 
systems  to  detect  disease  outbreaks. 

Objective:  To  evaluate  whether  data  on  microbiologic  laboratory  tests  ordered  for 
patients  during  outpatient  visits  to  MTFs  can  improve  the  performance  of  DoD- 
ESSENCE  in  detecting  disease  outbreaks. 

Specific  Aims: 

(1)  Identify  microbiology  laboratory  tests  for  which  frequency  of  ordering 
increases  during  disease  outbreaks,  and  frequency  of  ordering  follows  similar  patterns  to 
frequency  of  outpatient  visits  for  disease  syndromes  monitored  by  DoD-ESSENCE. 


(2)  Evaluate  and  compare  strategies  for  using  test  orders  for  syndromic 
surveillance,  alone  and  in  combination  with  outpatient  visit  data. 

Study  Design:  Secondary  analysis  of  electronic  medical  records  database. 

Relevance:  Early  and  reliable  detection  and  intervention  can  reduce  the  consequences  of 
disease  outbreaks. 

Results:  Related  laboratory  test  orders  can  be  combined  into  syndromes  that  align 
closely  both  with  existing  surveillance  using  outpatient  data,  and  with  CDC  expert  panel 
recommendations.  Sensitivity,  specificity,  and  timeliness  of  surveillance  using 
laboratory-based  respiratory  and  gastrointestinal  syndrome  data  are  similar  to 
surveillance  using  outpatient  visit  data.  Combining  the  data  sources  may  lead  to 
increased  timeliness  of  outbreak  detection  and  improve  the  performance  of  DoD- 
ESSENCE. 

Conclusion:  Data  on  laboratory  test  orders,  currently  collected  and  archived  for 
administrative  purposes,  may  be  useful  as  a  supplementary  data  source  for  syndromic 
surveillance. 
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CHAPTER  I:  Introduction 


Syndromic  surveillance  systems  monitor  one  or  more  electronic  data  sources  in 
real  time  to  assist  in  early  detection  of  unusual  health  events.  To  detect  such  events  at 
military  treatment  facilities  (MTFs),  the  Department  of  Defense  Electronic  Surveillance 
System  for  the  Early  Notification  of  Community-based  Epidemics  (DoD-ESSENCE) 
conducts  daily  surveillance  on  outpatient  visit  diagnosis  and  pharmacy  data.  Combining 
data  from  multiple  sources  may  improve  the  ability  of  syndromic  surveillance  systems  to 
detect  disease  outbreaks. 

The  purpose  of  this  study  is  to  evaluate  whether  data  on  microbiologic  and  serologic 
laboratory  tests  ordered  for  patients  by  providers  during  outpatient  visits  to  military 
treatment  facilities  (MTFs)  can  improve  the  performance  of  DoD-ESSENCE  in  detecting 
disease  outbreaks.  Improved  performance  will  be  measured  by  sensitivity,  specificity 
and  timeliness  of  outbreak  detection.  This  chapter  will  describe  the  specific  aims  and 
public  health  significance  of  the  study. 

Specific  Aims 

The  first  specific  aim  is  to  identify  laboratory  tests  that  may  be  associated  with 
disease  outbreaks,  and  with  other  measures  of  disease  prevalence.  We  will  identify 
laboratory  tests  that  meet  the  criteria  outlined  below: 

•  The  laboratory  test  is  more  likely  to  be  ordered  during  an  outpatient  visit  in  which 
the  patient  is  diagnosed  with  a  disease  syndrome  under  surveillance  by 
ESSENCE,  than  during  an  outpatient  visit  in  which  no  such  syndrome  is 
diagnosed. 
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•  Daily  counts  of  the  laboratory  test  are  positively  correlated  over  time  with  daily 
counts  of  outpatient  visits  for  one  or  more  of  the  disease  syndromes  under 
surveillance  by  ESSENCE. 

•  Daily  counts  of  the  laboratory  test  are  higher  during  outbreaks  of  a  disease 
syndrome  under  surveillance  by  DoD-ESSENCE  than  during  non-outbreak 
periods. 

We  will  propose  individual  laboratory  tests,  or  syndrome  groupings  comprised  of 
several  related  laboratory  tests,  for  surveillance 

The  second  specific  aim  is  to  develop  and  compare  several  strategies  for  using 
laboratory  test  orders  for  syndromic  surveillance,  alone  and  in  combination  with 
outpatient  visit  data.  Surveillance  strategies  will  be  developed  for  one  or  more  laboratory 
tests  or  syndromes  identified  under  the  first  specific  aim.  Components  of  the  second 
specific  aim  include: 

•  Identify  appropriate  statistical  model(s)  for  estimating  the  expected  daily  number 
of  laboratory  tests,  based  on  the  distribution  of  the  observed  data. 

•  Identify  and  implement  one  or  more  methods  for  combining  laboratory  test  order 
data  with  outpatient  visit  data  for  surveillance. 

•  Compare  these  models  and  methods  with  respect  to  timeliness,  sensitivity,  and 
specificity  of  outbreak  detection,  and  identify  the  most  promising  strategy. 

Public  Health  Significance 

The  stated  goal  of  this  project  is  to  improve  the  performance  of  a  syndromic 
surveillance  system  for  early  detection  of  outbreaks.  The  public  health  significance  of 
the  project,  therefore,  is  contingent  upon  the  importance  of  early  outbreak  detection  in 
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protecting  public  health,  and  the  potential  of  the  data  to  improve  early  detection.  This 
section  will  summarize  these  two  points,  and  discuss  expected  conclusions  of  the  study. 

In  theory,  early  detection  of  outbreaks  is  important  because  it  gives  public  health 
officials  time  to  intervene  to  prevent  the  spread  of  disease  and  reduce  morbidity  and 
mortality.  Early  detection  of  disease  is  only  useful  if  an  intervention  exists  that,  if 
administered  early,  can  reduce  morbidity  and  mortality.  The  course  of  many  infectious 
diseases  includes  a  nonspecific  prodrome  in  which  symptoms  of  the  disease  may  be 
confused  with  other  diseases,  and  during  which  intervention  may  save  the  life  of  the 
patient.  Anthrax,  for  example,  has  a  prodrome  lasting  from  several  hours  to  several  days 
[1],  and  administering  antibiotics  during  this  period  can  protect  against  the  fatal 
consequences  of  the  fully-developed  disease.  While  the  relatively  short  prodrome 
increases  the  likelihood  that  the  first  case  will  be  identified  by  a  clinician  before 
syndromic  surveillance  triggers  an  investigation  [1],  syndromic  surveillance  can  be  used 
to  discern  the  size  and  geographic  spread  of  the  outbreak  and  identify  potential  cases. 

Smallpox  has  a  prodrome  lasting  7  to  19  days,  and  administration  of  the  smallpox 
vaccine  during  the  first  four  days  after  exposure  is  protective  [2],  An  effective  syndromic 
surveillance  system  can  be  an  important  tool  for  public  health  officers  and  can  trigger 
early  investigations  and  interventions  to  both  minimize  disease  severity  and  to  halt 
further  person-to-person  transmission. 

Since  the  anthrax  attacks  of  2001,  several  articles  have  been  written  about  the 
importance  of  early  intervention.  Brookmeyer  and  Blades  [3]  estimated  that  in  the 
absence  of  antibiotic  prophylaxis,  the  number  of  deaths  resulting  from  the  anthrax  attacks 
would  have  doubled.  An  economic  model  developed  by  Kaufmann  et  al  [4]  showed  that 
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the  single  most  important  means  of  reducing  losses  after  an  anthrax,  tularemia,  or 
brucellosis  attack  is  post  attack  prophylaxis,  and  that  the  earlier  the  prophylaxis  program 
is  initiated,  the  greater  the  savings. 

The  earlier  an  outbreak  is  detected,  the  earlier  public  health  interventions  can  be 
initiated.  Wagner  et  al.  [5]  outline  four  methods  for  improving  timeliness  of  detection: 

(1)  improving  the  quality  of  existing  data  sources,  (2)  adding  new  data  sources,  (3) 
improving  the  detection  algorithm,  and  (4)  reducing  the  specificity  of  the  detection 
algorithm.  The  reason  that  adding  new  data  sources  can  improve  timeliness  is  that  it  can 
reinforce  the  outbreak  “signal”  relative  to  the  “noise”  in  the  data.  The  result  is  analogous 
to  increasing  the  sample  size. 

We  expect  to  show  that  laboratory  test  orders  may  be  combined  into  syndrome 
groups  similar  to  those  proposed  by  the  CDC,  and  that  these  syndrome  groups  will 
correlate  well  with  ICD-9  based  syndromes  for  outpatient  visit  data.  The  syndrome 
groups  may  be  used  by  others,  and  the  classification  algorithm  developed  during  this 
study  may  be  used  as  a  starting  point  for  a  free-text  classification  algorithm  for 
prospective  data  collection. 

We  also  anticipate  that  combined  surveillance  using  both  laboratory  test  orders 
and  outpatient  visit  data  will  show  better  performance  in  the  outbreak  detection 
evaluation  than  surveillance  using  either  data  set  alone.  In  this  way,  the  results  of  the 
study  can  be  used  to  improve  the  timeliness  of  outbreak  detection,  increasing  the 
likelihood  of  early  intervention. 


CHAPTER  II:  Background  and  Significance 


This  section  will  discuss  syndromic  surveillance,  place  it  within  the  context  of 
disease  surveillance  in  general,  and  discuss  important  aspects  of  syndromic  surveillance 
including  data  sources,  case  definitions,  algorithms,  and  evaluation  methods  used  for 
syndromic  surveillance. 

Syndromic  surveillance 

Syndromic  surveillance  systems  have  been  developed  as  a  way  to  reduce  the 
consequences  of  disease  outbreaks  through  early  detection  and  intervention.  Generally, 
these  systems  monitor  health-related  events  that  precede  diagnosis,  such  as  visits  to  a 
primary  care  provider  or  emergency  department,  or  medication  prescriptions.  An 
unexpected  increase  in  these  events  triggers  an  alert  and  subsequent  outbreak 
investigation.  Syndromic  surveillance  can  trigger  an  investigation  earlier  in  the  course  of 
the  disease  outbreak  than  traditional,  diagnosis-based  surveillance.  Earlier  investigation 
can  lead  to  earlier  public  health  interventions,  including  (1)  identifying  and  containing 
the  source  of  the  outbreak,  (2)  working  to  prevent  illness  in  exposed  persons,  and  (3) 
identifying  and  treating  cases  of  disease  early  in  their  course,  thus  saving  lives  and 
resources. 

No  uniform  definition  of  syndromic  surveillance  has  been  adopted.  We  list  five 
representative  definitions  from  the  syndromic  surveillance  literature: 

•  A  spectrum  of  activities  that  include  monitoring  illness  syndromes  or  events, 
such  as  medication  purchases,  that  reflect  the  prodromes  of  bioterrorism- 
related  diseases  [1]. 
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•  The  surveillance  of  disease  syndromes  (groups  of  signs  and  symptoms),  rather 
than  specific,  clinical,  or  laboratory-defined  diseases  [6]. 

•  An  investigational  approach  where  health  department  staff,  assisted  by 
automated  data  acquisition  and  generation  of  statistical  alerts,  monitor  disease 
indicators  in  real-time  or  near  real-time  to  detect  outbreaks  of  disease  earlier 
than  would  otherwise  be  possible  with  traditional  public  health  methods  [7]. 

•  The  monitoring  of  available  data  sources  for  outbreaks  of  unspecified  disease 
or  of  specified  disease  before  identifying  symptoms  are  confirmed  [8]. 

•  The  ongoing,  systematic  collection,  analysis  and  interpretation,  and 
application  of  real-time  (or  near-real-time)  indicators  for  diseases  and 
outbreaks  that  allow  for  their  detection  before  public  health  authorities  would 
otherwise  note  them.  Syndromic  surveillance  is  distinguished  from  other 
methods  of  surveillance  by  the  data  types  that  are  monitored  as  potential 
indicators  of  a  disease  or  outbreak  [9]. 

Each  of  these  definitions  emphasizes  one  or  more  important  aspects  of  syndromic 
surveillance:  data  sources  (automated  electronic  data),  case  definitions  (groups  of  signs 
and  symptoms  rather  than  laboratory-based  diagnoses),  and  outcomes  (statistical  alerts 
that  may  signal  disease  outbreaks).  These  aspects  will  be  discussed  in  detail  below. 

The  goal  of  syndromic  surveillance  has  been  defined  as  “to  enable  earlier 
detection  of  epidemics  and  a  more  timely  public  health  response,  hours  or  days  before 
disease  clusters  are  recognized  clinically,  or  before  specific  diagnoses  are  made  and 


reported  to  public  health  authorities”  [1],  or  according  to  Burkom  et  al.  [8],  “To 
complement  existing  sentinel  surveillance  by  identifying  outbreaks  with  false-alert  rates 
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acceptable  to  the  public  health  infrastructure.”  A  high  priority  for  syndromic  surveillance 
systems  is  early  detection  of  outbreaks.  However,  practical  experience  has  shown  that 
syndromic  surveillance  may  also  be  useful  for  estimating  the  magnitude  of  a  health 
problem,  documenting  the  distribution  and  spread  of  a  health  event,  evaluating  control 
and  prevention  measures,  and  detecting  changes  in  health  practice.  For  example,  during 
the  SARS  outbreak  of  2003,  syndromic  surveillance  was  used  to  reassure  public  health 
officials  that  the  disease  was  not  widespread  in  the  United  States  [10].  After  former 
President  Clinton’s  cardiac  bypass  surgery  in  September  2004,  syndromic  surveillance 
systems  documented  a  change  in  patient  behavior,  as  indicated  by  an  increase  in  patients 
seeking  care  for  chest  pain  [11],  Montgomery  County,  Maryland  used  syndromic 
surveillance  to  determine  when  to  begin  and  end  an  influenza  vaccination  program  [12], 
Finally,  when  outbreaks  are  detected,  either  through  syndromic  surveillance  or  traditional 
methods,  the  electronic  data  used  for  syndromic  surveillance  can  be  used  to  identify  and 
locate  patients  with  signs  and  symptoms  of  the  illness,  so  that  they  can  be  interviewed 
and  tested  during  an  epidemiologic  investigation. 

Syndromic  surveillance  has  been  effective  at  detecting  naturally  occurring 
outbreaks  such  as  food-borne  or  water-borne  outbreaks  of  gastrointestinal  disease,  and 
seasonal  outbreaks  of  respiratory  disease  [13].  It  also  appears  promising  for  detection  of 
disease  outbreaks  due  to  bioterrorist  attacks,  although  its  true  effectiveness  cannot  be 
determined  in  the  absence  of  such  an  attack.  In  addition,  even  if  an  outbreak  is  first 
detected  by  other  means,  syndromic  surveillance  provides  a  mechanism  for  monitoring 
the  outbreak  in  near-real  time  that  does  not  add  to  the  burden  of  healthcare  providers. 
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One  of  the  earliest  examples  of  a  syndromic  surveillance  system  is  ESSENCE,  the 
Electronic  Surveillance  System  for  the  Early  Notification  of  Community-based 
Epidemics.  Researchers  at  the  Walter  Reed  Army  Institute  of  Research  (WRAIR) 
developed  ESSENCE  initially  to  detect  infectious  disease  outbreaks  at  military  treatment 
facilities  in  the  National  Capital  Area.  Following  the  events  of  September  11,  2001, 
surveillance  was  expanded  to  cover  active  duty  personnel  and  beneficiaries  at  more  than 
300  military  treatment  facilities  worldwide.  ESSENCE  monitors  outpatient  visits  by 
grouping  International  Classification  of  Diseases,  9th  Revision  (ICD-9)  diagnostic  codes 
into  syndrome  groups.  Baseline  levels  of  outpatient  visits  in  these  groups  have  been 
established,  and  fluctuations  are  monitored  on  a  daily  basis.  When  a  significant  increase 
is  detected,  an  outbreak  investigation  may  be  initiated.  A  related  system,  ESSENCE  II, 
was  developed  in  collaboration  with  the  Johns  Hopkins  University  Applied  Physics  Lab, 
and  monitors  both  civilian  and  military  data  sources  in  the  Washington,  D.C.  region. 
ESSENCE  II  is  described  in  detail  in  Lombardo  et  al.  [12],  This  study  deals  exclusively 
with  the  military  version  of  ESSENCE,  referred  to  below  as  “DoD-ESSENCE”. 

Syndromic  surveillance  has  several  important  limitations.  First,  it  may  only  be 
useful  for  detecting  particular  sizes  and  types  of  outbreaks.  Buehler  et  al.  [1]  suggest  that 
syndromic  surveillance  is  most  likely  to  detect  an  outbreak  earlier  than  it  would  be 
detected  by  clinical  reporting  if  the  distribution  of  the  incubation  period  is  narrow,  the 
disease  has  a  long  prodrome,  there  is  an  absence  of  specific  clinical  signs  during  the 
prodrome,  and  diagnosis  is  unlikely  during  routine  care.  Anthrax,  because  of  its  variable 
incubation  period  and  relatively  short  prodrome,  is  likely  to  be  detected  clinically  before 
enough  cases  occur  to  trigger  an  alert  through  syndromic  surveillance.  Smallpox,  on  the 
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other  hand,  has  a  long,  nonspecific  early  phase  that  may  be  detected  by  syndromic 
surveillance.  There  is  no  way  to  predict  whether  naturally-occurring  emerging  infections, 
or  known  agents  modified  by  bioterrorists,  would  have  characteristics  that  would  lead  to 
early  detection  through  syndromic  surveillance.  Syndromic  surveillance  is  a  useful 
adjunct  to,  not  a  replacement  for,  clinician  reporting  and  other  traditional  surveillance 
methods. 

A  second  limitation  is  the  inherent  tradeoff  among  sensitivity,  specificity,  and 
timeliness.  Early  syndromic  surveillance  systems  were  prone  to  frequent  false  alarms. 
Systems  may  be  designed  to  improve  specificity  (and  reduce  false  alarms)  by  setting  a 
higher  threshold  for  alerts,  but  this  reduces  their  ability  to  detect  real  outbreaks,  and  to 
detect  them  early  [14].  The  only  way  to  improve  one  aspect  of  performance  without 
decreasing  other  aspects  of  performance  is  to  change  the  system,  possibly  by  adding  new 
data  sources  or  improving  detection  algorithms. 

Finally,  the  output  of  a  syndromic  surveillance  system  is  a  statistical  alert.  In 
order  for  syndromic  surveillance  to  have  any  effect  on  public  health,  it  must  be  integrated 
with  the  public  health  response  system  so  that  an  epidemiologic  investigation  and 
appropriate  public  health  response  can  take  place.  While  Stoto  et  al.  consider  this  a 
limitation  of  syndromic  surveillance,  it  is  in  fact  a  characteristic  of  all  surveillance 
systems.  It  is  critical,  however,  that  syndromic  surveillance  systems  be  designed  for  ease 
of  use  by  public  health  practitioners,  so  that  system  use,  and  public  health  benefits,  can  be 


maximized. 


10 


Disease  surveillance 

Public  health  surveillance  has  been  defined  as  the  “ongoing,  systematic  collection, 
analysis  and  interpretation  of  health  data  essential  to  the  planning,  implementation,  and 
evaluation  of  public  health  practice,  closely  integrated  with  the  timely  dissemination  of 
these  data  to  those  who  need  to  know.  The  final  link  of  the  surveillance  chain  is  the 
application  of  these  data  to  prevention  and  control.  A  surveillance  system  includes  a 
functional  capacity  for  data  collection,  analysis,  and  dissemination  linked  to  public  health 
programs”  [15].  Within  this  definition,  different  approaches  to  public  health  surveillance 
are  distinguished  by  their  goals,  uses  and  data  sources.  Syndromic  surveillance  focuses 
specifically  on  early  detection  of  outbreaks,  but  as  discussed  above,  may  be  useful  for 
other  purposes  as  well. 

Parrish  and  McDonnell  [16]  divide  data  collection  activities  for  surveillance  into 
two  categories:  primary  and  secondary.  Primary  data  collection  includes  interviews, 
such  as  completion  of  a  death  certificate  after  speaking  with  the  deceased’s  next-of-kin; 
and  observation,  such  as  the  physical  exam  portion  of  the  National  Health  and  Nutrition 
Examination  Study.  These  methods  are  costly  and  time  consuming  for  both  the 
participant  and  the  interviewer/examiner.  Secondary  data  collection,  on  the  other  hand, 
involves  existing  records  or  data  collected  for  another  purpose,  such  as  records  review. 
Secondary  data  collection  is  generally  faster  and  less  expensive  than  primary  data 
collection,  but  since  the  data  were  not  specifically  collected  for  surveillance,  the  quality 
is  likely  to  be  lower.  Syndromic  surveillance  is  an  example  of  secondary  data  collection. 
Since  timeliness  is  critical  for  outbreak  detection,  the  lower  quality  of  the  data  is  accepted 
as  a  necessary  trade-off  for  faster  data  collection. 
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Surveillance  systems  are  often  classified  into  active  and  passive  systems.  Active 
systems  require  public  health  personnel  to  seek  out  cases  by  contacting  health  care 
providers,  while  passive  systems  rely  on  health  care  providers  to  send  reports  periodically 
to  health  departments  [17].  Syndromic  surveillance,  however,  combines  the  ease  of 
passive  surveillance  with  the  capture  of  active  surveillance.  Syndromic  surveillance 
typically  uses  records  that  are  already  collected  for  other  purposes  such  as  billing  or 
medical  records.  An  automated  electronic  transfer  system  is  usually  set  up  so  that  neither 
the  health  care  provider  nor  public  health  personnel  must  actively  seek  cases.  Public 
health  personnel  are  typically  responsible  for  monitoring  syndromic  surveillance  data 
over  time. 

Electronic  data  sources 

Syndromic  surveillance  “gathers  information  about  the  group  of  symptoms 
experienced  by  cases  during  the  early  phase  of  illness”  [6].  Information  on  these 
symptoms  may  be  obtained  directly  from  clinical  information,  or  indirectly  from 
surrogate  data  sources.  Clinical  information  sources  commonly  used  for  syndromic 
surveillance  include  emergency  department  chief  complaints,  hospital  admissions, 
outpatient  diagnoses,  pharmacy  prescriptions,  911  calls,  and  nurse  hotline  calls  [6,  18], 
Various  systems  incorporate  non-clinical  data,  such  as  school  or  work  absenteeism  and 
over-the-counter  medication  sales,  as  well  [12,  19]. 

Syndromic  surveillance  systems  rely  on  electronic  data  sources  for  several 
reasons.  First  is  the  importance  of  timeliness  in  outbreak  detection.  Electronic  data  can 
be  transmitted  and  analyzed  more  quickly  than  traditional  pencil-and-paper  or  phone 
reporting.  For  example,  Mostashari  et  al.  [20]  showed  that  annual  influenza  outbreaks  in 
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New  York  City  are  typically  detected  two  to  three  weeks  earlier  by  syndromic 
surveillance  based  on  ambulance  dispatch  calls  than  by  traditional  sentinel  physician 
surveillance.  The  fact  that  syndromic  surveillance  using  electronic  data  uses  data  that  are 
already  generated  for  other  purposes  means  that  it  does  not  add  to  the  burden  of 
providers,  either  in  time  or  cost.  The  timeliness  of  electronic  data,  however,  depends  on 
how  quickly  providers  enter  and  transmit  the  data.  In  DoD-ESSENCE,  for  example, 
most  of  the  outpatient  data  is  available  for  surveillance  within  one  to  three  days  of  a 
patient  encounter  [21].  Immediate  data  entry,  transmission,  and  analysis  would  improve 
the  timeliness  of  outbreak  detection. 

Laboratory  test  order  data  is  a  promising  data  source  for  improving  syndromic 
surveillance  that  has  been  recommended  by  several  authors  [6,  7].  Pavlin  et  al.  [22]  point 
out  that  while  laboratory  tests  are  ordered  by  clinicians  and  may  reflect  actual  illness 
patterns,  tests  are  not  ordered  for  all  (or  most)  patients,  so  may  not  provide  as  complete  a 
picture  of  disease  patterns  as  other  data  sources.  However,  laboratory  tests  may 
complement  other  data  sources  in  important  ways.  Since  laboratory  tests  may  be  ordered 
for  the  sickest  patients,  surveillance  of  laboratory  test  data  may  be  more  specific  for 
severe  disease.  When  a  diagnosis  cannot  be  made  at  the  initial  visit,  the  types  of  tests 
ordered  may  provide  more  information  about  the  patient’s  symptoms  than  is  provided  by 
other  information  contained  in  electronic  records,  such  as  ICD-9  diagnostic  codes. 
Surveillance  conducted  on  laboratory  tests  as  they  are  ordered,  rather  than  as  the  results 
are  obtained,  may  give  an  early  indication  of  disease  outbreaks. 

No  other  study  has  examined  the  use  of  laboratory  test  orders  for  syndromic 
surveillance.  Although  laboratory  test  order  data  have  been  proposed  as  a  data  source  by 
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several  authors,  such  data  are  not  widely  available.  The  CDC  began  receiving  data  in 
2004  from  a  civilian  laboratory,  LabCorp®,  which  conducts  laboratory  tests  nationwide. 
Although  the  CDC  has  proposed  syndrome  definitions  and  has  begun  conducting 
surveillance  on  laboratory  test  orders  [23],  they  have  not  conducted  an  evaluation  to 
validate  the  syndrome  definitions  or  to  determine  whether  laboratory  test  orders 
contribute  to  the  detection  of  outbreaks.  In  addition,  their  data  and  target  population  are 
different  from  those  considered  in  this  study. 

A  few  studies  have  used  electronic  laboratory  test  results  for  surveillance.  Effler 
et  al.  [24]  showed  that  electronic  laboratory  reports  are  more  timely  and  complete  than 
conventional  reports,  and  a  review  by  Bravata  et  al.  [25]  indicated  that  automated 
laboratory  test  results  detect  76  -  100%  of  illness  identified  by  traditional  reporting 
methods.  Koski  et  al.  [26]  showed  that  data  from  a  commercial  lab  (the  Quest 
Diagnostics  data  archive)  could  be  used  for  influenza  surveillance.  Ma  et  al.  [27]  showed 
that  seasonal  and  geographic  patterns  of  West  Nile  Virus  were  well-represented  in 
laboratory  test  order  data.  Hutwagner  et  al.  [28]  published  one  of  the  earliest  studies  of 
electronic  surveillance,  showing  that  electronic  laboratory  reports  of  Salmonella  isolates 
were  reasonably  sensitive  and  specific  for  detecting  Salmonella  outbreaks,  and  were 
instrumental  in  early  detection  of  an  international  Salmonella  outbreak  spread  by 
contaminated  alfalfa  seeds  in  1995. 

Wagner  et  al.,  in  a  report  commissioned  by  the  Agency  for  Healthcare  Research 
and  Quality  (AHRQ),  suggested  that  data  from  clinical  laboratory  systems  would  be  a 
good  data  source  for  surveillance  because  most  laboratories  are  highly  automated  and 
report  results  electronically  [29]  .  Unfortunately,  DoD  laboratory  test  results  are  not 
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reported  in  a  standardized  fonn  and  are  not  likely  to  available  quickly  enough  to  provide 
early  indication  of  disease  outbreaks,  so  they  will  not  be  considered  in  this  study. 

Only  one  published  research  study  to  date  has  analyzed  DoD  laboratory  test  data. 
Riegodedios  et  al.  [30]  combined  laboratory  test  data  with  inpatient  and  outpatient  visit 
records  for  patients  diagnosed  with  one  of  four  reportable  diseases  (malaria,  syphilis, 
acute  hepatitis  B,  and  Lyme  disease)  to  determine  how  many  of  these  patients  had  a 
confirmatory  laboratory  test  result.  Overall,  they  found  that  only  19  percent  of  patients 
with  an  inpatient  diagnosis  code  corresponding  to  one  of  these  diseases,  and  only  16 
percent  of  patients  with  an  outpatient  diagnosis  code  corresponding  to  one  of  these 
diseases,  had  a  confirmatory  laboratory  test  result.  The  study  concluded  that  monitoring 
of  inpatient  and  outpatient  visit  records  alone  would  produce  many  false  positive  reports. 

An  unpublished  presentation  [31]  showed  that  DoD  laboratory  tests  may  be  used  to 
“track  antibiotic  resistance  and  antibiotic  resistant  infections,  provide  an  initial  and  rapid 
analysis  of  medical  event  concerns,  initiate  epidemiologic  investigations  as  needed,  and 
generate  hypotheses  for  further  study.”  DoD  laboratory  data  have  proven  useful  for 
surveillance  of  specific  pathogens,  including  group  a  beta-hemolytic  Streptococcus 
pyogenes,  methycillin-resistant  Staphylococcus  aureus  (MRSA),  and  Acinetobacter 
baumannii  [32-34].  This  study  complements  the  ongoing  research  by  Riegodedios  et  al. 
by  focusing  specifically  on  syndromic  surveillance. 

Case  definition  and  syndrome  grouping 

Syndromic  surveillance  typically  monitors  syndromes,  not  specific  illnesses. 

These  syndromes  are  defined  by  signs  and  symptoms  rather  than  by  laboratory-confirmed 
diagnoses.  Monitoring  syndromes  can  lead  to  earlier  outbreak  detection  for  two  reasons: 
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first,  many  illnesses  have  similar  signs  and  symptoms  during  early  stages  of  illness,  so 
syndromes  can  be  recognized  earlier  than  specific  diseases;  and  second,  laboratory 
confirmation  of  diagnoses  can  take  a  week  or  more. 

DoD-ESSENCE  currently  monitors  the  following  syndromes:  botulism-like, 
fever,  gastrointestinal,  hemorrhagic,  neurologic,  rash,  respiratory,  and  shock-coma. 

Syndromes  are  defined  on  the  basis  of  ICD-9  diagnostic  codes  assigned  to  patients  during 
outpatient  visits  to  MTFs.  Detailed  definitions  of  the  DoD-ESSENCE  syndromes  are  in 
Appendix  3.  These  syndrome  definitions  were  confirmed  against  the  gold  standard  of 
physician  chart  review.  Across  three  syndromes  (respiratory,  gastrointestinal  and  fever) 
and  three  hospital  emergency  departments,  sensitivities  ranged  from  67-95%  and 
specificities  ranged  from  92-97%  [13]. 

The  CDC  convened  a  working  group  in  2004  to  develop  a  preliminary  assignment 
of  laboratory  tests  to  syndrome  categories  similar  to  those  used  by  ESSENCE.  A  group 
of  clinicians  involved  with  syndromic  surveillance  were  given  a  list  of  the  laboratory 
tests  for  which  the  CDC  obtains  LabCorp  data,  and  asked  to  assign  each  laboratory  test  to 
one  or  more  syndromes.  The  committee  compiled  the  assignments  into  a  single  list, 
discussed  and  resolved  disagreements,  and  presented  a  preliminary  grouping  of 
laboratory  tests  into  syndromes  at  the  2004  Syndromic  Surveillance  Conference  [35]. 

The  grouping  is  presented  in  Appendix  1,  and  is  used  as  a  reference  for  classifying 
laboratory  tests  in  the  DoD  data  set. 

Once  syndromes  are  defined,  the  next  step  is  to  develop  a  rule  for  assigning 
individual  data  records  to  syndrome  groups.  Unlike  numeric  ICD-9  codes,  laboratory  test 
orders  are  recorded  as  text  in  the  DoD’s  electronic  database.  When  categorizing  free  text 
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fields,  it  is  important  to  account  for  the  different  terms,  spellings,  and  abbreviations  used 
by  different  providers  and  data  entry  personnel.  Two  basic  approaches  to  text  coding 
have  been  applied  to  syndromic  surveillance:  Bayesian  classification  and  keyword 
classification.  The  Real-Time  Outbreak  Detection  System  (RODS)  laboratory  at  the 
University  of  Pittsburgh  has  developed  a  Bayesian  classification  algorithm  for  assigning 
emergency  department  chief  complaints  to  syndromes  [36],  The  first  step  in  developing 
this  algorithm  was  to  produce  a  “training  set”  of  nearly  30,000  chief  complaint  entries 
that  were  manually  assigned  to  syndromes  by  a  physician.  The  algorithm  reads  new  chief 
complaint  entries,  compares  them  with  the  entries  in  the  training  set,  and  determines  the 
probability  that  the  new  entry  will  fall  into  each  syndrome,  based  on  its  similarity  to 
entries  in  the  training  set  and  the  syndromes  to  which  the  training  set  entries  are  assigned. 
The  new  entry  is  assigned  to  the  syndrome  with  the  highest  probability. 

The  New  York  City  Department  of  Health  and  Mental  Hygeine  (DOHMH)  uses  a 
different  approach  to  classifying  emergency  department  chief  complaints  [37].  Their 
system  searches  for  keywords  in  each  chief  complaint  entry,  and  if  one  or  more  keywords 
assigned  to  a  syndrome  are  present  in  the  entry,  the  entry  is  assigned  to  that  syndrome.  A 
recent  study  showed  that  the  two  methods  had  a  moderate  level  of  agreement  (kappa  = 
0.614)  when  used  to  categorize  the  same  data  [38].  The  civilian  version  of  ESSENCE 
uses  a  form  of  keyword  matching  that  is  described  by  Sniegoski  [39], 

Both  methods,  however,  are  likely  to  misclassify  some  cases  with  unique 
spellings  or  abbreviations.  Ideally,  all  laboratory  tests  could  be  classified  by  hand.  In 
this  study,  because  we  will  be  analyzing  a  finite,  retrospective  data  set,  and  because  many 
laboratory  test  names  (e.g.,  THROAT  CUL)  are  used  repeatedly,  we  can  classify 
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laboratory  test  orders  by  hand.  The  classification  we  develop  may  be  useful  in  the  future 
to  identify  keywords  for  a  classifier  similar  to  the  one  used  by  the  New  York  City 
DOHMH,  or  to  use  as  a  training  set  for  a  Bayesian  classifier  similar  to  the  one  used  by 
RODS. 

Algorithms  for  alerting 

Syndromic  surveillance  systems  generally  monitor  daily  counts  of  health-related 
events  over  time  and  look  for  aberrations.  Algorithms  for  aberration  detection  include 
three  steps:  (1)  Estimate  the  expected  count,  (2)  Compare  the  expected  count  with  the 
observed  count,  (3)  Signal  an  alert  if  the  observed  count  is  significantly  larger  than  the 
expected  count.  An  alert,  however,  does  not  signify  that  a  disease  outbreak  has 
necessarily  occurred,  only  that  an  unusual  pattern  has  been  observed  in  the  data.  An 
investigation  is  required  to  determine  the  cause  of  the  unusual  pattern. 

Expected  daily  counts  are  estimated  from  recent  data  using  methods  from  several 
different  disciplines,  including  statistics,  quality  control,  and  epidemiology.  Most 
statistical  approaches  are  regression  models  of  the  basic  form: 

V,  =  Xf>  +  e, 

where  Yt  is  the  observed  count  at  time  t,  X,  is  a  vector  of  characteristics  of  the  current 
time  period,  f  is  a  vector  of  regression  coefficients,  and  et  is  an  independent  and 
identically  distributed  error  term.  The  Xt  vector  often  includes  such  characteristics  as 
season  and  day  of  the  week,  to  account  for  cyclical  patterns  in  the  data.  If  counts  are 
large,  et  may  be  assumed  to  follow  a  normal  distribution  [40],  However,  it  is  likely  that 
the  ets  are  not  independent  because  of  serial  correlation  in  the  data.  Standard  time  series 
regression  models  may  be  used  in  this  situation.  Reis  et  al.  [41],  for  example,  proposed 
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an  autoregressive  moving  average  ( ARMA)  model,  in  which  the  expected  count  in  the 
current  time  period  is  estimated  as  the  weighted  sum  of  previous  counts  and  previous 
residuals  (differences  between  previous  observed  and  expected  counts). 

If  the  counts  are  small,  it  may  be  unreasonable  to  assume  normally  distributed 
residuals,  and  Poisson  regression  may  be  more  appropriate  than  linear  regression.  If 
counts  are  sufficiently  small,  even  sporadic,  serial  correlation  is  not  an  issue  [40]. 

Kleinman  et  al.  [42]  propose  using  generalized  linear  mixed  models,  of  which  Poisson 
regression  is  a  subset,  for  surveillance,  and  in  fact  Poisson  regression  is  used  for 
syndromic  surveillance  in  the  Boston  region  [43]. 

One  potential  drawback  of  regression  methods  is  that  they  require  a  substantial 
amount  of  historical  data  for  model  estimation  and  prediction.  The  CDC  recommends  at 
least  three  years  of  historical  data  if  seasonal  trends  are  to  be  estimated  [44],  Most 
syndromic  surveillance  systems  began  after  September  2001  and  have  limited  historical 
data.  In  addition,  automating  regression  models  to  monitor  a  variety  of  syndromes  and 
data  sources  is  problematic  because  the  models  often  require  fine  tuning  to  achieve  a 
good  fit  to  the  data  [40], 

Quality  control  methods,  on  the  other  hand,  typically  require  only  recent  data  for 
prediction.  These  methods  have  been  adapted  from  manufacturing,  where  they  are  used 
to  ensure  that  manufacturing  processes  stay  within  specified  limits.  Two  such  methods, 
exponentially  weighted  moving  average  (EWMA)  and  cumulative  sums  (CUSUM)  are 
used  in  syndromic  surveillance. 

EWMA,  also  known  as  exponential  smoothing,  was  first  applied  to  monitoring 
surveillance  data  by  Ngo  et  al.  [45]  for  detecting  nosocomial  outbreaks.  The  predicted 
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count  at  the  current  time  period  is  a  weighted  average  of  counts  at  previous  time  periods, 
with  greater  weights  given  to  more  recent  time  periods.  The  general  formula  is 

yt+i  =  ay, +  ( 1-  a)  y, 

where  the  parameter  a  is  chosen  to  minimize  the  forecast  error  variance,  y,  -  y,. 

EWMA  is  appropriate  for  independent  and  identically  distributed  data  series.  If 
there  is  a  secular  trend  or  a  seasonal  pattern,  two  adjacent  time  points  are  likely  to  have 
more  similar  counts  than  two  distant  time  points.  Some  days,  such  as  weekends  and 
holidays,  are  likely  to  have  lower  average  values  because  clinics  are  closed  or  fewer 
appointments  are  available  on  those  days.  These  systematic  changes  in  the  data  series 
over  time  can  seriously  affect  the  accuracy  of  forecasts  [40].  However,  if  systematic 
differences  in  the  mean  can  be  removed  by  regression,  EWMA  may  be  used  to  monitor 
changes  in  the  residuals. 

CUSUM  methods  add  up  the  deviations  between  observed  and  expected  values 
over  time.  The  CUSUM  following  time  t  is 

S,  =  max(0,  St-i  +  z  -  k) 

where  z  =  value  at  time  t,  normalized  to  a  mean  of  0  and  a  variance  of  1,  and  k  is  a 
parameter  that  is  often  chosen  to  be  equal  to  0.5,  or  one  half  the  standard  deviation  of  the 
normalized  the  values.  Therefore,  only  values  more  than  0.5  standard  deviations  above 
the  expected  value  are  accumulated  in  the  CUSUM.  An  alert  is  signaled  if  S,  >  h,  where 
h  is  a  threshold  value  chosen  to  balance  sensitivity,  specificity  and  timeliness.  After  an 
alert,  the  CUSUM  is  reset  to  zero  and  restarted. 

A  potential  disadvantage  of  the  CUSUM  approach  is  that  it  assumes  that  the  data 
follow  a  normal  distribution  and  that  observations  are  not  serially  correlated.  Like 
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EWMA,  CUSUM  assumes  that  there  are  no  systematic  changes  in  the  mean  daily  count 
over  time.  Rogerson  and  Yamada,  however,  describe  an  extension  of  CUSUM  that 
assumes  the  data  follow  a  Poisson  distribution,  and  allows  for  seasonal  effects  [46]. 

Both  CUSUM  and  EWMA  were  designed  to  detect  small  changes  in  average 
daily  counts.  Frisen  demonstrated  that  EWMA  yields  the  minimal  expected  delay  in 
detection  for  a  fixed  false  alarm  probability,  and  CUSUM  yields  the  minimal  expected 
delay  in  detection  for  the  “worst”  history  of  observations  before  the  outbreak  occurred 
[47].  It  is  not  clear  which  method  is  optimal  for  syndromic  surveillance,  and  ESSENCE 
offers  users  the  option  of  a  CUSUM-based  algorithm  in  addition  to  EWMA. 

Scan  statistics  have  been  used  by  epidemiologists  to  detect  disease  clusters 
retrospectively  for  several  decades  [40].  Kulldorff  [48]  extended  the  method  for 
prospective  surveillance,  and  developed  public  domain  software  for  this  purpose  called 
SatScan  (available  at  www.satscan.org).  SatScan  can  be  used  to  conduct  surveillance  for 
outbreaks  over  time  in  a  specific  geographic  region,  but  it  can  also  be  used  to  identify 
clusters  of  disease  in  space  across  geographic  regions. 

SatScan  counts  the  number  of  events  (e.g.  outpatient  visits,  laboratory  tests 
ordered)  that  took  place  within  time  interval  d  of  the  current  time  t,  and  compares  the 
count  to  its  expected  value  under  a  Poisson  distribution.  The  scan  statistic  is  defined  as 
the  largest  likelihood  ratio  across  all  sets  of  distances  d  for  which  the  count  is  greater  than 
expected.  The  /;- value  for  the  scan  statistic  is  obtained  by  Monte  Carlo  simulation,  and 
an  alert  is  sounded  at  time  t  if  the  p-v alue  falls  below  a  specified  threshold.  SatScan  has 
been  incorporated  into  many  syndromic  surveillance  systems,  including  ESSENCE  [8], 
the  New  York  City  DHMH  [49]  and  the  National  Bioterrorism  Syndromic  Surveillance 
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Demonstration  Program  in  Boston  [50],  mostly  for  spatial  or  spatio-temporal 
surveillance.  Spatial  surveillance  is  beyond  the  scope  of  this  dissertation;  we  will  focus 
on  detecting  temporal  clusters  within  geographic  locations  defined  by  MTFs  or  groups  of 
neighboring  MTFs. 

Other  surveillance  algorithms  have  been  proposed,  including  hidden  Markov 
models  [51],  wavelets  [36],  and  Bayes  belief  nets  [8].  So  far,  no  single  approach  has 
dominated  the  others  with  respect  to  its  ability  to  detect  outbreaks,  and  research  is 
ongoing.  One  study,  for  example,  compared  scan  statistics  with  time  series  regression 
and  spatial  regression,  and  found  that  while  the  scan  statistic  performed  best,  a 
“smorgasbord”  of  methods  was  better  than  any  individual  method  [52], 

DoD-ESSENCE  uses  a  combination  of  the  methods  described  above.  Linear 
regression  models  are  fit  to  data  from  the  previous  month  to  remove  time  trends  and 
weekend/holiday  effects.  If  counts  are  small  (typically,  median  <  5  per  day),  such  as  for 
a  small  MTF  or  a  rare  syndrome,  regression  models  are  likely  to  demonstrate  lack  of  fit 
to  the  data,  and  the  EWMA  algorithm  is  run  instead. 

Triggering  an  alert 

As  described  earlier,  syndromic  surveillance  systems  compare  observed  and 
expected  values,  and  trigger  an  alert  if  observed  values  exceed  expected  values  by  at  least 
a  specified  amount.  The  simplest  approach  to  setting  the  threshold  is  based  on  /7-values. 
An  alert  is  given  if  the  /7-value  for  the  difference  between  observed  and  expected  values 
is  less  than  0.05,  or  0.01.  This  corresponds  roughly  with  the  observed  value  being  more 
than  two  or  three  standard  deviations,  respectively,  above  the  expected  value. 
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However,  using  a  /?- value  of  0.05  or  0.01  will  yield  a  false  alert  every  20  or  100 
days  on  average.  This  concern  is  exacerbated  by  the  problem  of  multiple  comparisons. 
ESSENCE,  for  example,  monitors  seven  syndromes  in  each  of  more  than  300  MTFs 
every  day.  Even  if  the  threshold  for  each  syndrome  and  MTF  is/?  <  0.01,  DoD- 
ESSENCE  can  expect  to  see  several  false  alerts  every  day.  This  high  false  alert  rate 
places  an  undue  burden  on  those  who  investigate  alerts.  In  practice,  public  health 
personnel  may  not  initiate  an  investigation  unless  an  alert  is  sounded  two  days  in  a  row 
for  the  same  syndrome  and  MTF.  This  reduces  the  number  of  alerts  that  need  to  be 
investigated,  but  also  reduces  the  timeliness  of  the  investigation.  Programmers  can 
reduce  the  number  of  false  alerts  by  incorporating  multiple  comparisons  adjustments, 
such  as  the  Bonferroni  correction,  into  the  algorithm.  Again,  this  reduces  the  sensitivity 
and  timeliness  of  the  system. 

A  different  approach  is  to  set  no  explicit  threshold  for  alerting.  Some  systems 
simply  compile  a  daily  list  of  counts  ranked  by  their  /?-values,  and  note  how  many  days 
each  count  has  been  higher  than  expected.  Public  health  officials  can  scan  the  list  and 
use  their  judgment  to  determine  which  counts  to  investigate.  Advantages  of  this  system 
are  that  it  incorporates  human  judgment  and  it  is  acceptable  to  public  health  officials  who 
use  it,  since  they  retain  some  control  over  which  alerts  to  investigate. 

Finally,  an  explicitly  multivariate  approach  can  be  used  to  yield  a  combined  alert. 

Possible  multivariate  approaches  to  syndromic  surveillance  are  discussed  below. 

Multivariate  extensions 

A  characteristic  of  the  algorithms  described  so  far  is  that  they  are  used  to  monitor 
a  single  data  stream.  When  more  than  one  data  stream  is  monitored,  such  as  outpatient 
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visits  and  laboratory  test  orders  in  this  study,  the  false  alert  rate  will  increase.  Also,  a 
slight  increase  in  each  of  several  data  streams  might  not  trigger  an  alert  in  any  single  data 
stream,  but  if  the  increase  is  consistent  across  several  data  streams,  a  multivariate  method 
may  be  able  to  detect  it.  Several  approaches  have  been  considered,  including  parallel 
univariate  surveillance,  multivariate  quality  control  methods,  multivariate  regression,  and 
Bayes  belief  nets.  A  feature  of  all  of  these  methods  is  that  they  produce  a  single  /7- value 
for  the  entire  set  of  data  streams,  rather  than  an  individual  value  for  each  data  stream. 

Parallel  univariate  surveillance  involves  using  a  surveillance  algorithm  to  monitor 
each  data  stream  separately,  and  combining  the  results  to  decide  when  an  alert  should  be 
given.  Combinations  that  have  been  considered  are  the  maximum  /7-value,  the  product  of 
/7-values  or  the  mean  of  /7-values.  Using  the  maximum  /7-value  results  in  too  many  false 
alerts  [8],  Fisher  [53]  proposed  the  product  of  p- values,  which  follows  an  approximate 
chi  square  distribution.  Edgington  [54]  proposed  calculating  the  mean  of  /7-values,  which 
follows  an  approximate  normal  distribution,  and  claimed  that  the  mean  of  /7-values  is 
more  powerful  overall  than  the  product  of  /7-values.  Burkom  et  al.  found  that  both 
methods  performed  well  in  an  outbreak  detection  simulation  [55].  Fisher’s  method 
identified  many  alerts  that  were  present  in  a  single  data  stream,  while  Edgington’ s 
method  consistently  identified  combined  signals. 

Multivariate  control  charts  include  Hotelling’s  J2,  multivariate  CUSUM  and 
multivariate  EWMA  (MEWMA).  If  the  data  streams  follow  a  multivariate  normal 
distribution,  then  a  J2  control  chart  may  be  appropriate  [56],  T1  is  calculated  using  the 
following  formula: 

T1  =  n(y  -  y)'S  ‘(y  -  y) 
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where  y  is  the  vector  of  observed  counts,  y  is  the  vector  of  expected  counts,  and  S 
is  the  estimated  covariance  matrix  of  the  two  data  streams,  calculated  from  historical 
data. 

Multivariate  CUSUMs  may  be  calculated  in  several  different  ways.  The  simplest 
is  to  use  the  I2  statistic  as  the  data  for  calculating  the  CUSUM;  a  direct  multivariate 
generalization  of  the  univariate  CUSUM  was  proposed  by  Crosier  [57].  MEWMA, 
described  by  Lowry  et  al.  [58]  uses  the  formula: 

Yt+]=RYt  +  (\-R)  Yt 

where  Y,  is  the  vector  of  counts  at  time  t,  R  is  a  diagonal  weight  matrix,  and  /  is 
the  identity  matrix. 

Multivariate  CUSUM  and  MEWMA  have  been  shown  to  detect  changes  in  the 
average  value  of  the  data  stream  more  quickly  than  I2.  However,  all  three  of  these 
methods  were  found  by  Burkom  to  be  too  sensitive  when  used  for  syndromic  surveillance 
[55].  These  methods  were  likely  to  signal  alerts  after  changes  in  the  covariances  among 
data  streams,  even  if  no  change  in  the  average  daily  counts  occurred. 

Another  possible  approach  is  multivariate  regression.  Methods  for  estimating 
multivariate  time  series  and  multivariate  Poisson  regression  are  well  described  in 
statistical  texts  [59,  60].  The  generalized  linear  mixed  model  approach  proposed  by 
Kleinman  et  al.  [42]  to  borrow  strength  across  similar  geographic  regions  could  be 
adapted  to  borrow  strength  across  related  data  streams. 

ESSENCE  developers  have  also  explored  the  use  of  Bayes  belief  nets  [8].  This 
approach  estimates  the  joint  probability  distribution  of  all  available  data  and  estimates  the 
probability  of  an  outbreak  from  this  distribution.  The  ESSENCE  team  applied  Bayes 


25 


belief  nets  to  test  data,  compared  it  with  multivariate  and  parallel  univariate  control 
charts,  and  found  the  Bayes  belief  net  to  be  versatile  and  robust  [61].  Research  on  this 
method  is  ongoing.  At  present,  we  propose  to  use  parallel  univariate  surveillance 
methods,  but  will  monitor  Burkom’s  ongoing  research  and  consider  any  approach 
identified  as  promising. 

Outbreak  investigation 

The  final  step  in  syndromic  surveillance  is  investigating  alerts  to  determine 
whether  a  disease  outbreak  is  in  fact  occurring.  Investigation  may  begin  with  a  review  of 
the  data  that  triggered  the  alert,  and  may  extend  to  calling  hospitals  and  providers  for 
information,  or  a  full  review  of  all  identified  cases.  Some  alerts  may  be  resolved  easily, 
such  as  a  sudden  increase  in  outpatient  visits  for  Japanese  encephalitis  at  a  particular 
MTF  that  turned  out  to  be  a  systematic  miscoding  of  vaccinations  for  the  disease  [21], 
Others  require  more  extensive  follow-up,  such  as  outbreaks  of  gastrointestinal  illness  in 
which  laboratory  testing  and  food  recall  are  initiated.  The  New  York  City  DOHMH  has 
found  that  syndromic  surveillance  is  useful  for  citywide  increases  in  illness,  such  as  the 
annual  influenza  epidemic  or  large  outbreaks  of  norovirus,  but  that  small  localized 
outbreaks  are  often  missed  [37].  DoD-ESSENCE  detects  gastrointestinal  and  respiratory 
outbreaks  with  some  frequency,  typically  in  larger  MTFs  that  service  recruit  populations. 

Evaluating  syndromic  surveillance  systems 

Syndromic  surveillance  systems  are  relatively  new  and  their  utility  for  detecting 
different  sorts  of  disease  outbreaks  is  unproven.  If  resources  are  to  be  devoted  to 
syndromic  surveillance  systems,  it  is  important  to  establish  that  they  can  achieve  their 
stated  goals.  Also,  if  changes  are  to  be  made  to  existing  systems,  such  as  the  change 
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proposed  in  this  study,  it  is  important  to  establish  that  the  change  improves  the 
performance  of  the  system.  Since  the  stated  goal  of  syndromic  surveillance  is  early 
outbreak  detection,  systems  should  be  evaluated  with  respect  to  ability  to  detect  existing 
outbreaks  (sensitivity)  and  to  do  so  earlier  than  other  methods  (timeliness).  Because  it  is 
always  possible  to  improve  sensitivity  and  timeliness  by  increasing  the  false  alert  rate 
(reducing  specificity),  and  frequent  false  alerts  reduce  the  utility  of  syndromic 
surveillance,  specificity  should  be  evaluated  as  well. 

The  CDC  has  established  a  framework  for  evaluating  syndromic  surveillance  that 
addresses  these  aspects  of  system  performance  [7].  Evaluating  sensitivity,  specificity  and 
timeliness  requires  data  on  well-defined  outbreaks.  For  naturally  occurring  outbreaks, 
historical  data  may  be  used.  Some  outbreaks  of  interest  for  surveillance,  such  as  those 
caused  by  bioterrorism,  are  not  available  in  historical  data  and  must  be  simulated. 
Outbreaks  may  be  simulated  by  superimposing  a  simulated  outbreak  on  authentic 
background  data,  or  by  simulating  both  outbreaks  and  background  data.  A  final  approach 
to  evaluation  is  to  superimpose  authentic  outbreaks  on  simulated  background  data.  These 
approaches  are  outlined  in  Mandl  et  al.  [62], 

We  use  historical  data  including  real  outbreaks  for  this  evaluation  .  An  advantage 
of  this  approach  is  that  no  assumptions  must  be  made  about  the  shape  of  the  epidemic 
curve  or  the  distribution  of  the  background  data.  One  potential  disadvantage  is  that 
outbreaks  due  to  bioterrorism  may  not  resemble  naturally  occurring  outbreaks,  so  the 
ability  of  a  surveillance  system  to  detect  naturally  occurring  outbreaks  may  not  extend  to 
its  ability  to  detect  bioterrorist  attacks.  However,  early  symptoms  of  some  pathogens 
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likely  to  be  used  by  bioterrorists  may  match  symptoms  of  respiratory  and  gastrointestinal 
outbreaks  [63], 

In  the  evaluation  framework,  once  outbreaks  are  defined,  the  syndromic 
surveillance  algorithm  is  run  on  data  that  contain  these  outbreaks.  For  each  outbreak,  it  is 
determined  whether  an  alert  was  signaled,  which  days  of  the  outbreak  were  identified, 
and  on  which  day  the  alert  was  first  given.  Sensitivity  and  specificity  may  be  calculated 
across  all  outbreaks,  or  across  all  outbreak  days,  using  the  standard  formulas.  Timeliness 
is  often  defined  as  the  mean  (or  median)  number  of  days  between  the  first  day  of  the 
defined  outbreak  and  the  first  alert. 

Timeliness  depends  in  part  on  how  quickly  the  following  events  take  place:  an 
exposed  patient  seeks  medical  care;  the  provider  orders  laboratory  tests;  the  laboratory 
tests  are  entered  into  the  electronic  database,  transmitted  to  the  central  server,  and 
analyzed;  and  the  results  are  used  to  inform  public  health  investigations.  A  tabulation  of 
laboratory  test  orders  shows  that  for  90  percent  of  completed  microbiology  laboratory 
tests,  specimens  were  collected  on  the  same  day  that  the  test  was  ordered.  Because 
laboratory  tests  in  the  DoD  system  must  be  entered  into  an  electronic  database  before  the 
specimen  is  tested,  data  could  be  transmitted  and  analyzed  in  near  real  time.  Whether  or 
not  laboratory  test  orders  provide  timely  indication  of  disease  outbreaks  may  depend 
more  on  how  often  and  how  early  in  the  course  of  disease  the  laboratory  tests  are  ordered. 

If  laboratory  tests  are  only  ordered  in  later  stages  of  disease,  then  surveillance  using 
laboratory  test  order  data  may  still  be  useful  for  confirming  and  investigating  outbreaks, 
but  is  unlikely  on  its  own  to  lead  to  early  detection.  Comparing  the  timeliness  of 
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surveillance  using  two  data  sources,  laboratory  test  orders  and  outpatient  visits,  can  shed 
light  on  this  question. 

An  aspect  of  validity  that  is  not  directly  addressed  in  the  CDC’s  framework  is  the 
validity  of  syndrome  definitions.  In  other  words,  are  we  monitoring  the  right  laboratory 
tests  to  identify  patients  with  signs  and  symptoms  of  particular  syndromes?  The 
syndrome  definitions  proposed  by  the  CDC’s  working  group  (Appendix  1)  have  face 
validity  in  that  experts  think  that  the  tests  in  each  syndrome  are  likely  to  be  ordered  when 
the  patient  presents  with  signs  and  symptoms  associated  with  that  syndrome.  To 
establish  criterion  validity,  we  compare  laboratory  test  orders  against  outpatient  visit 
volume  for  the  same  syndrome.  Discriminant  validity  will  be  based  on  low  correlations 
with  other  syndromes,  at  least  relative  to  correlation  with  the  assigned  syndrome. 

Syndromic  surveillance  systems  must  also  be  evaluated  in  practice:  Are  they 
acceptable  to  public  health  workers  who  use  them?  Are  they  flexible,  portable,  and 
stable?  Are  they  cost  effective?  We  believe  that  adding  laboratory  test  order  data  to 
DoD-ESSENCE  would  not  greatly  affect  system  operating  characteristics.  ESSENCE  II, 
a  version  of  ESSENCE  that  combines  military  and  civilian  data  sources  for  surveillance 
in  the  Washington,  DC  area,  has  been  evaluated  with  respect  to  these  criteria  [12].  This 
study  uses  retrospective  analysis  to  assess  validity,  and  leaves  prospective  studies  of 
system  experience  for  future  research. 

Summary 

This  chapter  has  described  the  definition  and  goals  of  syndromic  surveillance, 
data  sources,  syndrome  definitions,  alerting  algorithms,  and  evaluation  methods.  This 
study  uses  a  new  data  source,  develops  and  validates  syndrome  definitions  for  this  data 
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source,  applies  existing  surveillance  algorithms  using  this  data  set,  and  evaluates  the 
results.  The  next  chapter  will  discuss  the  methodology  in  detail. 


CHAPTER  III:  Materials  and  Methods 


As  stated  above,  the  purpose  of  this  study  is  to  evaluate  whether  data  on 
microbiology  laboratory  tests  ordered  for  patients  by  providers  during  outpatient  visits  to 
military  treatment  facilities  (MTFs)  can  improve  the  performance  of  ESSENCE  in 
detecting  disease  outbreaks.  Improved  performance  is  measured  by  sensitivity, 
specificity  and  timeliness  of  outbreak  detection.  This  section  details  the  materials  and 
methods  used  in  the  study,  including  data  sources,  statistical  analysis,  and  evaluation 
criteria. 

Description  of  data 

Data  for  this  project  consist  of  administrative  records  obtained  from  two  sources: 
laboratory  test  records  from  the  Executive  Information  and  Decision  Support  Program 
Office  (EI/DS)  of  TRICARE,  and  outpatient  visit  data  from  DoD-ESSENCE.  EI/DS  has 
provided  data  on  microbiology  laboratory  test  orders,  excluding  HIV  tests,  from 
outpatient  MTFs  worldwide.  Data  are  available  from  November  2002  to  November 
2004.  WRAIR  biostatisticians  provided  access  to  DoD-ESSENCE  automated  ambulatory 
military  health  system  visit  data  for  the  same  time  period. 

The  target  population  for  this  study  is  all  active  duty  military,  military  retirees, 
and  family  members  who  are  eligible  to  receive  care  in  military  treatment  facilities. 
Subjects  represent  all  age  groups,  both  male  and  female.  No  power  analysis  was 
performed,  since  we  obtained  all  records  rather  than  a  sample  of  records.  The  complete 
data  set  contains  records  for  3.4  million  microbiology  laboratory  tests  on  1.2  million 
patients. 
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We  characterize  the  data  with  respect  to  person  (age,  sex,  branch  of  service, 
relationship  to  military  sponsor),  place  (military  treatment  facility  and  clinic  type),  and 
time  (weekly  and  seasonal  patterns,  and  time  lags  between  test  order,  specimen 
collection,  and  test  result). 

Figure  3.1  illustrates  the  process  by  which  a  laboratory  test  is  archived  in  the 
database.  Because  test  information  can  be  “lost”  at  any  point  in  the  process,  the  archive 
of  data  used  in  this  study  does  not  include  all  test  orders.  While  it  is  impossible  to 
ascertain  exactly  how  many  tests  are  ordered  but  never  archived,  discussions  with 
laboratory  personnel  suggest  the  number  of  such  tests  is  low  relative  to  the  total  volume 
of  tests.  Uncommon  laboratory  tests  may  be  sent  to  non-DoD  laboratories,  but  if  they 
were  ordered  within  the  DoD  system,  they  should  be  entered  into  the  database  when  the 
results  are  returned.  Laboratory  tests  ordered  for  DoD  beneficiaries  by  civilian  providers 
will  not  be  archived  in  the  database  unless  the  tests  are  sent  to  a  DoD  laboratory. 

However,  because  TRICARE  provides  better  coverage  for  tests  performed  in  a  DoD 
laboratory,  it  is  likely  that  many  tests  ordered  by  civilian  providers  are  sent  to  DoD  labs. 

This  study  will  treat  the  archived  laboratory  tests  as  if  they  represent  all  tests  ordered,  and 
analyze  them  based  on  the  date  of  order  rather  than  the  date  on  which  they  were  archived. 
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Figure  3.1:  DoD  Laboratory  Ordering  Process 


Coding  laboratory  test  descriptions 

The  DoD  does  not  currently  use  a  standard  reporting  system  for  laboratory  test 
orders.  Laboratory  ordering  and  reporting  systems  are  developed  by  region,  and  within  a 
region  the  reporting  system  may  require  the  provider  to  select  the  appropriate  laboratory 
test  from  a  list,  or  may  allow  free  text.  As  a  result,  a  single  type  of  test  may  be  reported 
in  many  different  ways,  including  multiple  spellings  and  abbreviations.  For  example,  in  a 
preliminary  sample  of  the  DoD  data,  rapid  strep  tests  were  indicated  in  at  least  six 
different  ways:  'RAP  STRE',  'RAP  STREP',  'RAPID  ST',  'RAPID  STREP',  'RAPID 
STREP  A',  and  'RAPSTP&C'. 

We  will  develop  a  standard  nomenclature  to  account  for  different  spellings  and 
abbreviations.  First,  test  names  will  be  grouped  using  a  list  of  common  synonyms. 
Second,  test  names  that  do  not  appear  on  the  list  of  synonyms  will  be  assigned  by  hand  to 
an  existing  category,  or  a  new  category  will  be  created.  Third,  test  orders  for  which  the 
test  name  contains  insufficient  information  will  be  assigned  to  a  category  based  on 
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specimen  source  where  possible.  For  example,  a  “miscellaneous  culture”  for  which  the 
specimen  source  is  “stool”  will  be  considered  a  “stool  culture.”  Finally,  multiple  tests  for 
the  same  illness  will  be  combined  into  a  single  category. 

Subject  identification  and  human  use 

Access  to  the  data  extracted  for  use  in  this  project  was  provided  through  a 
password  protected  network  drive  shared  within  the  security  of  the  WRAIR  firewall. 

Data  were  provided  to  WRAIR  after  removal  of  all  identifying  information  by  EI/DS 
programmers.  Records  for  a  subject  may  be  linked  using  an  encrypted  pseudo-identifier 
unique  to  the  subject;  however,  the  research  team  does  not,  and  will  not,  have  the 
password  to  reverse  the  encryption,  thus  cannot  obtain  any  identifying  information  on 
individual  subjects.  The  data  analysis  was  conducted  at  WRAIR,  USUHS  or  from  the 
Pi’s  personal  computer.  Data  are  reported  only  in  the  aggregate. 

In  addition  to  information  about  specific  laboratory  tests  and  diagnostic  codes,  the 
initial  data  files  contain  the  following  demographic  information  for  each  record:  age  (in 
years;  date  of  birth  will  not  be  on  the  file),  sex,  branch  of  service,  relationship  to  military 
sponsor,  MTF,  and  5 -digit  ZIP  code  (the  5 -digit  ZIP  code  is  not  used  in  this  analysis). 

Data  analysis 

This  study  relies  primarily  on  descriptive,  correlational  analysis,  along  with  time 
series  and  quality  control  methods.  SAS®  is  used  for  data  manipulation,  graphical 
comparisons  and  statistical  analyses. 

Validating  syndrome  definitions 

The  first  specific  aim  is  to  identify  laboratory  tests  that  may  be  associated  with 
disease  outbreaks,  and  with  other  measures  of  disease  prevalence.  Our  primary  measure 
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of  disease  prevalence  is  DoD-ESSENCE  syndrome  counts  based  on  outpatient  ICD-9 
codes.  The  data  used  to  construct  these  syndrome  counts  are  based  on  physician 
diagnoses  rather  than  patient  chief  complaint,  and  have  been  shown  to  correspond  well 
with  physician  assessments  of  disease  from  chart  reviews  [13].  We  examine  the 
association  between  the  CDC-defined  laboratory  test  syndromes  and  DoD-ESSENCE’s 
ICD-9  code  based  syndromes,  and  will  also  explore  associations  between  individual 
laboratory  tests  and  ICD-9  code  based  syndromes.  We  will  examine  the  association 
between  laboratory  test  orders  and  outpatient  visit  diagnoses  from  three  different 
perspectives. 

At  the  level  of  the  individual  outpatient  visit,  we  will  use  data  on  type  of 
laboratory  test  ordered  (from  laboratory  test  data)  and  ICD-9  diagnostic  codes  recorded 
during  the  outpatient  visit  in  which  the  test  was  ordered  (from  ESSENCE  outpatient  visit 
records).  Records  from  laboratory  test  data  will  be  linked  with  outpatient  visits  based  on 
pseudo  identifiers  and  date  of  visit/test  order.  We  will  identify  the  disease  syndromes 
associated  with  each  outpatient  visit  using  the  ICD-9  diagnostic  codes  for  that  visit  and 
DoD-ESSENCE  syndrome  definitions  (Appendix  3).  Contingency  tables  will  be 
constructed  for  each  syndrome  and  laboratory  test  of  interest,  with  cell  counts 
corresponding  to  the  number  of  outpatient  visits  in  each  category: 


Test  ordered? 

Syndrome  present? 

Yes 

No 

Yes 

a 

b 

No 

c 

d 

For  each  syndrome,  laboratory  tests  will  be  ranked  according  to  strength  of 
association  with  that  syndrome,  where  strength  of  association  is  measured  by  the  ratio  of 
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observed  number  of  visits  with  the  syndrome  in  which  the  test  was  ordered,  compared 
with  the  expected  number  if  laboratory  tests  ordered  were  independent  of  outpatient  visit 
diagnosis.  (The  expected  count  is  calculated  as  [a+b]*[a+c]/[a+b+c+d].)  Tests  that  are 
strongly  associated  with  each  syndrome  will  be  considered  for  surveillance  of  that 
syndrome. 

The  second  approach  will  examine  daily  volume  of  laboratory  test  orders  and 
outpatient  visits  over  time.  We  will  construct  daily  counts  for  each  laboratory  test  over 
the  period  of  the  study,  and  will  construct  daily  counts  of  outpatient  visits  for  each 
syndrome  over  the  same  time  period.  Spearman’s  rank  correlation  will  be  used  to 
measure  the  strength  of  association  over  time  between  each  laboratory  test  and  each  ICD- 
9  code  based  syndrome.  We  will  consider  overall  correlation  and  “residual”  correlation, 
after  removing  secular  trends,  seasonal  and/or  weekly  patterns  from  the  data  [19].  This 
will  allow  us  to  distinguish  between  correlation  due  to  similar  cyclical  patterns  of  disease 
and  health  care  seeking  behavior,  and  correlations  due  to  unexpected  changes  in  disease 
patterns.  For  each  syndrome,  laboratory  tests  will  be  ranked  according  to  strength  of 
association  with  that  syndrome,  and  tests  that  are  most  strongly  associated  with  each 
syndrome  will  be  considered  for  surveillance. 

To  determine  which  laboratory  tests  (or  combinations  of  laboratory  tests)  are  most 
strongly  associated  with  disease  outbreaks,  we  will  examine  daily  counts  of  laboratory 
test  orders  during  known  outbreaks.  The  strength  of  association  with  an  outbreak  will  be 
measured  by  the  ratio  of  the  average  daily  count  of  laboratory  tests  ordered  during  the 
outbreak  to  the  average  daily  count  of  laboratory  tests  ordered  during  a  comparable  non¬ 
outbreak  period,  such  as  a  period  of  the  same  length  immediately  preceding  the  outbreak. 
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Outbreaks  identified  by  ESSENCE  epidemiologists  during  the  period  from  November 
2002  to  November  2004,  in  MTFs  which  provided  E1L7  records  during  that  period,  will 
be  used.  Specific  outbreaks  are  listed  in  chapters  4  and  5  below. 

Evaluating  outbreak  detection  performance 

The  second  specific  aim  is  to  develop  and  compare  several  strategies  for  using 
laboratory  test  orders  for  syndromic  surveillance,  alone  and  in  combination  with 
outpatient  visit  data.  One  or  more  laboratory  tests  or  syndromes  identified  under  the  first 
specific  aim  will  be  used  for  this  analysis.  Below  we  outline  and  discuss  the  components 
of  the  surveillance  system  to  be  used. 

Surveillance  algorithm :  Several  off-the-shelf  algorithms  are  available  for 
syndromic  surveillance.  We  propose  to  use  the  DoD-ESSENCE  algorithm  in  order  to 
make  the  results  of  the  study  most  relevant  for  DoD-ESSENCE  planners  in  the  future. 

We  also  consider  the  CUSUM  algorithms  implemented  in  the  CDC’s  EARS,  a  temporal 
scan  statistic,  and  a  simple  EWMA  algorithm,  since  these  algorithms  are  freely  available 
to  the  public. 

The  DoD-ESSENCE  algorithm  is  described  above,  and  is  discussed  in  detail 
elsewhere  [8,  21].  The  two  main  components  of  this  algorithm  are  regression  and 
EWMA.  In  the  regression  model,  day  of  the  week,  holiday,  and  time  trend  will  be 
considered  as  independent  variables,  and  the  dependent  variable  will  be  daily  counts.  We 
will  also  use  the  EWMA  algorithm  currently  in  use  by  the  DoD-ESSENCE  system, 
including  the  parameters  that  are  specified  by  this  algorithm.  Finally,  we  will  examine 
whether  regression  models  tend  to  fit  the  data  well,  or  if  running  the  EWMA  algorithm 
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on  the  raw  data  is  generally  sufficient.  Performance  will  be  evaluated  as  described 
below. 

Threshold  for  alerting :  In  general,  as  the  threshold  for  alerting  is  raised,  false 
alarms  will  be  less  frequent  (increased  specificity)  but  smaller  outbreaks  are  more  likely 
to  be  missed  (decreased  sensitivity)  and  slow-building  outbreaks  will  be  caught  later  in 
their  development  (decreased  timeliness).  For  evaluation  purposes  it  is  not  necessary  to 
specify  a  single  threshold  for  alerting;  instead,  the  tradeoffs  among  sensitivity,  specificity 
and  timeliness  will  be  explored  in  detail. 

Combining  data  sources :  It  is  possible  that  laboratory  test  order  data  are  entered 
and  transmitted  more  quickly  than  outpatient  data  under  the  current  electronic  system.  In 
a  version  of  CHCS  that  is  still  under  development,  however,  laboratory  test  data  will  be 
entered  and  transmitted  along  with  outpatient  and  pharmacy  data  in  a  single  record.  For 
this  reason,  we  do  not  expect  future  laboratory  test  order  data  to  provide  an  earlier  signal 
than  these  data  sources  when  an  outbreak  occurs.  However,  laboratory  test  order  data 
may  strengthen  the  ratio  of  signal  to  noise  to  improve  the  probability  of  detecting 
outbreaks.  To  take  advantage  of  this  possibility,  we  will  look  at  laboratory  test  order  data 
in  combination  with  outpatient  visit  data.  Parallel  univariate  surveillance  algorithms  will 
be  run  separately  on  outpatient  visits  and  laboratory  test  data,  and  /^-values  will  be 
combined  to  obtain  an  overall  /;- value  comparing  observed  events  to  expected  events. 

We  will  consider  the  methods  of  Edgington  [54]  and  Fisher  [53],  described  in  the 
background  section. 

Evaluate  the  strategy  with  respect  to  outbreak  detection  (sensitivity,  specificity  and 
timeliness ):  The  surveillance  algorithm(s)  will  be  evaluated  on  data  containing  known 
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outbreaks.  Two  sources  of  outbreaks  have  been  considered:  simulated  outbreaks,  in 
which  hypothetical  outbreaks  are  superimposed  onto  existing  data;  and  naturally 
occurring  outbreaks,  which  use  laboratory  test  order  data  for  dates,  MTFs  and  syndromes 
corresponding  to  a  set  of  known  outbreaks.  We  propose  to  use  data  on  naturally 
occurring  outbreaks  because  these  data  reflect  actual  patterns  of  illness,  not  hypothetical 
ones.  Furthermore,  simulating  realistic  outbreaks  in  laboratory  test  data  requires  a 
detailed  model  of  the  complex  relationships  among  exposure,  illness,  healthcare-seeking 
behavior,  and  laboratory  test  ordering  patterns.  The  proportion  of  outpatient  visits  in 
which  lab  tests  are  ordered,  the  specific  symptoms  for  which  each  test  is  likely  to  be 
ordered,  and  whether  ordering  behavior  changes  during  an  outbreak,  are  all  unknown,  so 
simulating  the  effects  of  an  outbreak  on  laboratory  test  orders  would  be  difficult. 

Unfortunately,  all  confirmed  outbreaks  identified  during  the  study  period  correspond 
to  respiratory  and  gastrointestinal  syndromes.  A  weakness  of  this  study  is  that  our  ability 
to  evaluate  sensitivity,  specificity  and  timeliness  is  limited  to  these  two  syndromes.  We 
will  use  case  studies  of  non-confirmed  outbreaks  to  describe  patterns  of  laboratory  test 
orders  for  these  syndromes.  Even  though  confirmed  outbreaks  have  not  been  identified 
for  all  syndromes,  this  research  will  provide  an  important  first  look  at  the  usefulness  of 
the  laboratory  test  data. 

We  will  evaluate  the  performance  of  the  algorithms  with  respect  to  timeliness, 
sensitivity  and  specificity.  The  surveillance  algorithm  will  be  run  on  the  data  with  known 
outbreaks  and  we  will  monitor  whether  (and  how  quickly)  the  algorithm  detects 
outbreaks,  and  how  often  the  surveillance  algorithm  triggers  a  false  alert.  Modified 
receiver  operating  characteristic  (ROC)  curves,  which  plot  sensitivity  against  background 
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alert  rate,  will  be  used  to  illustrate  the  tradeoff  between  sensitivity  and  specificity.  The 
background  alert  rate  is  defined  as  the  average  number  of  days  between  alerts  that  do  not 
correspond  to  a  verified  outbreak  (e.g.,  one  alert  every  six  weeks).  We  assume  that  the 
longer  the  period  between  background  alerts,  the  greater  the  specificity.  We  cannot 
measure  specificity  directly  because  some  alerts  may  correspond  to  true  outbreaks  that 
were  not  verified  by  public  health  personnel.  We  will  also  report  sensitivity  and 
timeliness  for  specific  background  alert  rates  ranging  from  one  alert  every  two  weeks  to 
one  alert  every  eight  weeks. 


CHAPTER  IV:  Development  and  evaluation  oflaboratory  test  nomenclature  and 


syndrome  definitions 


Introduction 

The  first  specific  aim  of  this  study  is  to  identify  laboratory  tests  that  may  be 
associated  with  disease  outbreaks,  and  with  other  measures  of  disease  prevalence.  We 
will  identify  laboratory  tests  that  meet  the  criteria  outlined  below: 

•  The  laboratory  test  is  more  likely  to  be  ordered  during  an  outpatient  visit  in  which 
the  patient  is  diagnosed  with  a  disease  syndrome  under  surveillance  by 
ESSENCE,  than  during  an  outpatient  visit  in  which  no  such  syndrome  is 
diagnosed. 

•  Daily  counts  of  the  laboratory  test  are  positively  correlated  over  time  with  daily 
counts  of  outpatient  visits  for  one  or  more  of  the  disease  syndromes  under 
surveillance  by  ESSENCE. 

•  Daily  counts  of  the  laboratory  test  are  higher  during  disease  outbreaks  than  during 
non-outbreak  periods. 

The  purpose  of  this  chapter  is  to  evaluate  laboratory  tests  with  respect  to  these 
criteria  and  propose  individual  laboratory  tests,  or  syndrome  groupings  comprised  of 
several  related  laboratory  tests,  for  surveillance. 

Description  of  data 

Data  consist  of  all  outpatient  microbiology  lab  tests  with  a  certified  result  that 
were  entered  into  the  CHCS  I  system  between  November  2,  2002  and  October  31,  2004. 
Data  include  tests  for  active  duty  military  personnel,  their  dependents,  and  retirees  from 
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all  branches  of  service  at  MTFs  worldwide.  Characteristics  of  laboratory  tests  and  the 
patients  for  whom  they  were  ordered  are  described  in  Table  4.1. 


Table  4.1:  Characteristics  of  laboratory  tests  and  patients  used  in  the  analysis 


%  of  tests 

%  of  patients 

Sex 

Female 

67.8 

61.1 

Male 

32.2 

38.9 

Age  (years) 

0-4 

8.7 

9.0 

5-17 

15.7 

17.7 

18-64 

70.9 

69.2 

65+ 

4.7 

4.1 

Relationship  to  military  sponsor 

Child 

28.3 

30.2 

Sponsor 

34.9 

37.0 

Spouse 

36.6 

32.5 

Other 

0.2 

0.3 

Branch  of  service 

Army 

33.2 

33.5 

Air  Force 

24.6 

25.2 

Marine  Corps 

12.8 

13.0 

Navy 

27.0 

25.8 

Other 

2.4 

2.5 

Type  of  clinic 

Emergency 

17.2 

Primary  care 

62.3 

Other  outpatient 

20.5 

N 

1,825,194 

1,003,338 

The  CHCS  II  system  stores  laboratory  data  in  a  hierarchical  file  with  multiple 
records  for  each  lab  test.  Basic  information,  including  test  ordered,  specimen  source, 
MTF,  and  demographics  of  the  patient  are  repeated  in  each  record  for  a  single  test. 
Multiple  records  for  the  same  test  differ  primarily  with  respect  to  the  test  result;  for 
example,  growth  of  a  culture  may  be  observed  on  several  different  days,  with  one  record 
for  each  observation;  or  sensitivity  to  several  antibiotics  may  be  tested,  with  one  record 
for  each  antibiotic.  Since  the  test  results  will  not  be  used  for  syndromic  surveillance,  the 
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first  step  in  processing  the  data  was  to  collapse  the  file  so  that  it  contains  a  single  record 
for  each  laboratory  test. 

Next,  a  subset  was  selected  on  the  basis  of  date  and  MTF.  This  analysis  included 
tests  ordered  from  November  2002  through  October  2004.  Of  325  land-based  MTFs  that 
recorded  at  least  one  laboratory  test  in  the  CHCS  I  database,  112  either  did  not  start 
reporting  until  after  June  2004,  or  had  significant  gaps  in  reporting  during  the  November 
2002-October  2004  time  frame.  The  analysis  sample  is  restricted  to  the  remaining  214 
land-based  MTFs  with  complete  data  (Appendix  2).  Completeness  was  determined 
through  examination  of  time  series  plots  of  daily  laboratory  test  order  counts  for  each 
MTF,  and  tabulation  of  first  and  last  order  dates  for  each  MTF.  One  MTF  with  complete 
data,  the  USS  Eisenhower,  will  not  be  included  because  it  is  the  only  ship  that  submitted 
laboratory  records. 

Data  on  outpatient  visits  to  MTFs  were  obtained  for  comparison  purposes.  Visit 
date,  MTF,  patient  age  and  sex,  ICD-9  diagnostic  codes,  and  encrypted  identifiers  for  the 
patient  and  provider  were  obtained  from  CHCS  standard  outpatient  data  records  (SADR). 

All  outpatient  visits  to  the  214  included  MTFs  during  the  study  period  were  selected. 

Each  outpatient  visit  was  evaluated  for  assignment  to  one  or  more  syndrome  categories 
based  on  recorded  ICD-9  codes.  DoD-ESSENCE  syndrome  definitions  from  January 
2004  were  used  for  syndrome  assignment  (Appendix  3). 

Test  order  standardization 

Different  MTFs  use  different  abbreviations  to  describe  the  same  test.  For 


example,  sputum  cultures  were  recorded  as  “SP  CULTURE”,  “SPU  CULT”,  “SPUT 
C&S/SMEAR”,  or  simply,  “SPUTUM”.  A  total  of  870  distinct  test  order  names  appear 
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in  the  laboratory  test  data  archive  from  November  2002  through  October  2004.  These 
were  categorized  into  71  standard  test  names,  using  the  following  process: 

•  Dr.  Julie  Pavlin  obtained  from  Major  Martin  Tenney  a  standardized  list  of  DoD 
microbiology  lab  tests  (Appendix  4).  Major  Tenney  is  part  of  an  initiative  to 
standardize  lab  test  orders  in  CHCS.  He  also  provided  a  list  of  common 
synonyms  for  the  lab  tests  on  the  standardized  list.  Overall,  73%  of  the  lab  tests 
descriptions  in  the  laboratory  data  match  his  standardized  list  or  one  of  the 
synonyms  he  provided. 

•  Major  Tenney’s  list  included  separate  categories  when  different  tests  could  be 
ordered  for  the  same  illness.  For  example,  in  his  list,  ANTHRAX  PHAGE, 

ANTHRAX  CULTURE,  ANTHRAX  DFA,  AND  B  ANTHRACIS  ID  are  all 
listed  as  separate  tests.  Drs.  Julie  Pavlin,  Shilpa  Hakre,  and  J.D.  Malone  reviewed 
the  list  and  recommended  which  categories  could  be  combined  for  the  purpose  of 
surveillance. 

•  Major  Tenney’s  list  does  not  include  large  categories  of  tests  that  are  represented 
in  the  data  set.  New  categories  were  created  for  acinetobacter,  acid-fast  bacilli, 
fecal  reducing  substances,  fecal  occult  blood,  fecal  white  blood  cells,  Group  A 
streptococcus,  Group  B  streptococcus,  herpes,  influenza,  blood  parasites  and 
leishmaniasis.  A  handful  of  tests  on  his  list,  such  as  tests  for 
Calymmatobacterium  or  Klebsiella  granulomatis  and  Coccidioides,  do  not  occur 


in  this  data  set  so  were  discarded. 
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•  Test  order  names  that  do  not  match  a  name  or  synonym  on  Major  Tenney’s  list 
were  reviewed  by  Dr.  Hakre  and  Dr.  Malone  and  assigned  to  one  of  his 
categories,  or  to  a  new  category  as  necessary. 

•  Some  test  order  names  contain  insufficient  information  to  assign  to  a  category, 
e.g.  MISC  CULTURE  or  BODY  FLD  CULTURE.  Records  corresponding  to 
these  test  order  names  were  examined  to  determine  the  source  of  the  specimen.  If 
the  specimen  source  was  urine,  blood,  cerebrospinal  fluid,  feces,  pharynx, 
sputum,  nose,  or  wound,  the  test  was  assigned  to  the  category  corresponding  to  a 
standard  culture  of  the  specimen. 

Appendix  5  summarizes  the  mapping  of  the  870  observed  test  names  to  71  standard  test 
names. 

Criterion  #1:  Laboratory  test/syndrome  co-occurrence 

Laboratory  test  data  and  outpatient  visit  data  were  merged  in  order  to  determine 
which  lab  tests  were  most  commonly  associated  with  which  diagnoses.  A  laboratory  test 
record  was  matched  to  an  outpatient  visit  record  by  encrypted  patient  identifier  (military 
sponsor’s  encrypted  SSN  and  patient’s  family  member  prefix)  and  date  (within  two 
days).  In  order  to  minimize  duplicate  matches,  the  following  steps  were  taken: 

•  Records  for  pending  laboratory  tests  were  excluded.  CHCS  contains  separate 
records  for  pending  and  final  test  results,  with  different  codes.  (26%  of  laboratory 
test  records  were  excluded  under  this  criterion). 

•  If  a  patient  saw  two  or  more  different  providers  for  the  same  syndrome  for  the 
same  day,  only  the  record  corresponding  to  the  first  visit  was  retained.  (9%  of 
outpatient  records  were  excluded  under  this  criterion.) 
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•  If  a  lab  test  was  matched  to  two  or  more  outpatient  visits  on  different  days,  the 
visit  date  closest  to  the  date  on  which  the  lab  test  was  ordered  was  retained. 

•  If  a  lab  test  was  matched  to  two  or  more  outpatient  visits  on  the  same  day,  one  of 
the  outpatient  visits  was  selected  at  random  for  inclusion. 

Outpatient  visits  and  tests  that  occurred  from  May  2003  through  April  2004  were 
included  in  this  analysis.  (Memory  limitations  on  the  secure  server  precluded  matching 
the  entire  two  years  of  data.)  A  total  of  18,579,731  records  were  examined.  Of  these,  0.7 
percent  (122,685  records)  correspond  to  a  laboratory  test  record  that  could  not  be 
matched  to  an  outpatient  visit.  Possible  explanations  are  that  the  laboratory  test  was 
ordered  more  than  two  days  after  the  visit,  it  was  not  ordered  in  conjunction  with  a  visit, 
or  the  data  were  entered  incorrectly.  Because  these  records  are  such  a  small  proportion  of 
the  data,  we  did  not  investigate  potential  explanations  further.  Eighty-six  percent 
(15,945,889  records)  correspond  to  an  outpatient  visit  that  was  neither  assigned  to  a 
syndrome  nor  matched  to  a  corresponding  laboratory  test.  These  include  routine  visits 
and  visits  with  no  diagnosis  of  infectious  disease.  Ten  percent  (1,870,977)  correspond  to 
a  visit  that  was  assigned  to  a  syndrome  on  the  basis  of  ICD9  codes  but  for  which  no 
laboratory  tests  were  ordered.  The  remaining  3.4  percent  (640,181  records)  correspond 
to  an  outpatient  visit  with  a  matched  laboratory  test. 

In  order  to  identify  laboratory  tests  that  were  most  likely  to  be  ordered  during 
outpatient  visits  for  each  syndrome,  we  crosstabulated  the  merged  laboratory  test  and 
outpatient  visit  file  by  laboratory  test  name  and  syndrome.  We  calculated  expected 
counts  in  each  cell  in  the  table  under  the  assumption  that  laboratory  test  orders  were  not 
associated  with  syndrome  diagnosis.  If  the  ratio  of  observed  to  expected  counts  in  a  cell 
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was  2.0  or  greater,  the  corresponding  laboratory  test  and  syndrome  were  determined  to  be 
associated.  Table  4.2  lists  laboratory  tests  associated  with  each  syndrome  under  this 
criterion. 

The  most  notable  pattern  is  that  aerobic  blood  cultures  and  unspecified  blood 
cultures  are  associated  with  all  syndromes  except  botulism-like  illness.  Ordering  patterns 
across  syndromes  are  similar  for  these  two  tests,  so  they  may  be  used  interchangeably  by 
different  MTFs.  Although  the  strongest  association  is  for  the  fever  and  shock/coma 
syndromes,  the  two  tests  combined  are  five  times  more  likely  to  be  ordered  when  any 
infectious  disease  syndrome  is  diagnosed  than  when  no  such  syndrome  is  diagnosed. 

Aerobic  blood  cultures  may  be  an  indicator,  therefore,  for  the  presence  of  any  infectious 
disease  but  may  not  differentiate  well  among  disease  syndromes. 
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Table  4.2:  Observed  vs.  expected  number  of  test  orders 


Syndrome 

std  test  ord 

Obs/exp 

ratio 

Syndrome 

std  test  ord 

Obs/exp 

ratio 

Bot  like 

BF  CULT 

5.49 

BLD  CULT 

4.73 

CSF  CULTURE 

4.77 

CSF  CULTURE 

471.93 

EAR  CULT 

3.75 

FUNGUS,  CSF 

600.63 

SPUTUM  CULT 

3.40 

FUNGUS,  OTHER 

7.32 

Fever 

AER  BLD  CULT 

28.53 

GRAM  STAIN 

134.80 

ANAER  BLD  CULT 

16.17 

HERPES 

12.87 

BLD  CULT 

20.24 

BLD  PARA 

21.66 

OTHER 

14.11 

BORDETELLA  CULT 

4.05 

VARICELLA 

337.09 

C  DIFFICILE 

6.05 

VIRAL  CULT 

17.91 

CSF  CULTURE 

19.90 

Rash 

AER  BLD  CULT 

5.60 

ECOLI  0157:H7 

10.79 

AEROBIC  CULT 

3.55 

EYE  CULT 

2.46 

ANAER  CULT 

4.56 

FECAL  WBC 

2.03 

BLD  CULT 

3.63 

FUNGUS,  CSF 

8.34 

BLD  PARA 

5.10 

GRAM  STAIN 

7.59 

FUNGUS,  OTHER 

2.62 

GROUP  A  STREP 

8.50 

GROUP  A  STREP 

4.11 

INFLUENZA 

25.27 

HERPES 

3.87 

OTHER 

2.33 

ROTAVIRUS 

7.05 

MISC  CULTURE 

6.33 

RSV 

10.78 

THR  CULT 

3.46 

SSC 

2.32 

TISSUE  CULT 

6.15 

STOOL  CULT 

2.35 

VARICELLA 

36.23 

THR  CULT 

6.55 

UA  CULT 

3.45 

VIRAL  CULT 

28.88 

VIRAL  CULT 

5.92 

WND  CULT 

4.58 

YERSINIA  CULT 

2.62 

GI 

AER  BLD  CULT 

3.70 

Resp 

AER  BLD  CULT 

3.54 

ANAER  BLD  CULT 

2.63 

ANAER  BLD  CULT 

2.56 

BLD  CULT 

3.05 

BLD  CULT 

3.46 

BLD  PARA 

3.98 

BORDETELLA  CULT 

8.03 

C  DIFFICILE 

28.51 

EAR  CULT 

5.38 

CMV 

5.15 

GROUP  A  STREP 

7.35 

CSF  CULTURE 

2.01 

INFLUENZA 

6.02 

ECOLI  0157:H7 

16.96 

FECAL  RS 

16.02 

RECTAL  CULT 

7.59 

FECAL  WBC 

43.10 

RESP  CULT 

4.62 

FOB 

2.02 

RESP  CULTURE 

5.44 

GIARDIA/CRYPTO 

13.33 

RSV 

7.31 

H  PYLORI  CULT 

12.55 

SPUTUM  CULT 

6.16 

O&P 

17.67 

OTHER 

2.91 

THR  CULT 

7.28 

OTHER  GI 

28.79 

VIRAL  CULT 

2.74 

ROTAVIRUS 

44.35 

Silk  Coma 

AER  BLD  CULT 

11.54 

SSC 

40.14 

AEROBIC  CULT 

2.44 

STOOL  CULT 

38.49 

BF  CULT 

16.36 

VIBRIO 

46.42 

BLD  CULT 

32.19 

YERSINIA  CULT 

14.41 

CSF  CULTURE 

7.12 

Hemr  ill 

AER  BLD  CULT 

8.02 

GRAM  STAIN 

6.23 

ANAER  BLD  CULT 

4.91 

O&P 

3.08 

ANAER  CULT 

2.42 

SPUTUM  CULT 

15.23 

H  PYLORI  CULT 

3.04 

STOOL  CULT 

3.57 

Neuro 

AER  BLD  CULT 

2.60 

UA  CULT 

2.27 

ANAER  BLD  CULT 

14.34 

WND  CULT 

2.10 

BF  CULT 

40.59 
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We  compared  strong  test/syndrome  associations  with  those  identified  by  a  CDC 
expert  panel  [23],  based  on  a  consensus  method  (Appendix  1).  Most  associations 
identified  by  the  CDC  expert  panel  were  also  observed  in  the  DoD  data  set.  However, 
there  were  several  exceptions: 

•  The  CDC  panel  associated  aerobic  and  anaerobic  cultures  (other  than  blood 
cultures)  and  MRSA  tests  with  fever,  but  those  associations  were  not  observed  in 
the  DoD  data.  Definitions  of  the  aerobic  and  anaerobic  cultures  may  differ  in  the 
two  data  sets;  in  this  case,  the  CDC  laboratory  data  do  not  seem  to  differentiate 
between  aerobic  cultures  of  blood  and  of  other  body  fluids.  Three-hundred 
ninety  MRSA  tests  were  ordered  in  the  DoD  data  set,  and  only  three  were 
associated  with  a  syndrome  diagnosis:  one  with  fever,  and  two  with  respiratory. 

•  The  CDC  panel  associated  viral  cultures  with  fever,  GI,  rash'  and  respiratory 
syndromes.  This  analysis  identified  associations  with  fever,  neurological,  rash, 
and  respiratory  syndromes  but  not  Gl  syndrome. 

•  The  CDC  panel  associated  cultures  of  Brucella,  Chlamydia,  respiratory  fungus, 
legionella,  and  mycoplasma  with  respiratory  syndrome,  but  these  associations 
were  not  observed  in  the  DoD  data.  The  total  number  of  Brucella,  Legionella 
and  Mycoplasma  cultures  reported  during  the  two  years  of  the  study  is  less  than 
10,  so  this  should  not  be  considered  evidence  against  the  existence  of  these 
associations.  Chlamydia  and  respiratory  fungus  cultures,  however,  were  ordered 
172  and  156  times,  respectively. 

This  analysis  identified  several  laboratory  test/syndrome  associations  that  were  not 
identified  by  the  CDC  expert  panel: 
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The  CDC  expert  panel  did  not  identify  any  associations  between  laboratory 
tests  and  the  botulism-like  illness  syndrome.  The  DoD  data  included  7,943 
diagnoses  of  botulism-like  illness  syndrome,  of  which  only  55  (<1%)  had  an 
associated  laboratory  test  order.  However,  within  this  small  subgroup, 
botulism-like  illness  was  associated  with  cultures  of  body  fluid,  cerebrospinal 
fluid,  ear,  and  sputum. 

Several  associations  with  the  fever  syndrome  did  not  correspond  to  the  CDC 
panel’s  recommendations,  including  bordetella,  c.  difficile,  E.  coli  0157:H7, 
fecal  WBC,  fungus  in  CSF,  group  A  strep,  influenza,  rotavirus,  RSV,  SSC, 
stool,  throat,  urine,  CSF,  eye,  and  Yersinia  cultures. 

Gastrointestinal  syndrome  was  associated  with  aerobic  and  anaerobic  blood 
cultures,  blood  parasite  tests,  CMV,  and  CSF  cultures. 

The  CDC  expert  panel  did  not  identify  any  associations  between  laboratory 
tests  and  the  hemorrhagic  illness  syndrome.  The  DoD  data  included  1 1,494 
diagnoses  of  hemorrhagic  illness  syndrome,  of  which  305  (2.7%)  were 
associated  with  a  laboratory  test  order.  Standard  blood  cultures  are 
associated  with  this  syndrome,  as  is  H.  pylori  culture.  However,  the  H.  pylori 
association  is  based  on  only  two  laboratory  test  orders. 

Most  of  the  tests  associated  with  neurological  illness  by  the  CDC  expert 
panel  do  not  appear  in  the  DoD  microbiology  data.  These  include  specific 
tests  for  West  Nile  Virus,  Fyme  disease,  and  several  types  of  encephalitis, 
which  may  be  recorded  with  chemistry  lab  tests.  However,  this  analysis 
indicates  that  neurological  syndrome  diagnoses  are  associated  with  viral, 
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blood  and  body  fluid  cultures;  Gram  stains;  and  tests  for  fungus  (other  than 
CSF,  genital  or  respiratory),  herpes,  and  varicella. 

•  The  microbiology  data  do  not  capture  the  tests  for  specific  illnesses  in  the 
rash  syndrome,  except  for  herpes  and  varicella.  However,  associations  are 
seen  between  the  rash  syndrome  and  several  general  tests  that  were  not 
identified  by  the  CDC  expert  panel.  These  include  blood  cultures,  blood 
parasites,  fungus  (other  than  CSF,  genital  or  respiratory),  group  A  strep,  and 
throat,  tissue,  viral,  wound  and  miscellaneous  cultures. 

•  Blood,  rectal  and  ear  cultures  are  the  only  associations  with  respiratory 
syndrome  that  were  observed  in  the  DoD  data  but  not  identified  by  the  CDC 
expert  panel. 

•  The  CDC  does  not  use  the  shock/coma  syndrome  defined  for  DoD- 
ESSENCE,  so  the  expert  panel  did  not  consider  associations  with  this 
syndrome.  This  analysis  indicates  that  Gram  stains,  ova/parasite  tests,  and 
cultures  of  blood,  body  fluids,  CSF,  sputum,  stool,  urine  and  wounds  are 
associated  with  this  syndrome.  Patients  presenting  with  this  syndrome  may 
be  very  ill  and  providers  may  be  inclined  to  order  any  and  all  general  tests  for 
these  patients. 

In  summary,  agreement  between  the  CDC  expert  panel  and  the  data  is  quite  high. 
Exact  agreement  cannot  be  calculated  since  the  two  data  sets  do  not  include  exactly  the 
same  list  of  tests.  This  analysis  identified  a  set  of  tests  that  seems  to  be  associated  with 
infectious  disease  in  general,  and  validated  associations  between  specific  tests  and 
syndromes. 
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Criterion  #2:  Time  series  correlations 

In  order  to  identify  laboratory  tests  that  are  correlated  over  time  with  particular 
syndromes,  we  chose  to  focus  on  data  from  six  regions.  Overall  time  series  correlations 
for  all  MTFs  combined  were  not  particularly  useful  because  there  is  so  much  variability 
across  MTFs.  We  chose  military  hospitals  from  different  regions  and  branches  of 
service,  along  with  their  associated  clinics,  in  order  to  explore  how  time  series 
correlations  might  vary  by  region,  service,  and  MTF  size. 

Fifty-six  MTFs  in  six  regions  were  selected  for  evaluation  under  criterion  #2. 
Each  region  consists  of  one  or  more  military  hospitals,  and  nearby  clinics  that  support  the 
hospitals.  These  MTFs  were  selected  from  among  the  214  land-based  MTFs  with 
complete  data  to  represent  different  geographic  locations  and  branches  of  service.  Table 
4.3  lists  the  hospitals  and  clinics  in  each  selected  region,  along  with  the  median  daily 
number  of  lab  tests  ordered  in  each. 


Table  4.3:  Description  of  regions  selected  for  time  series  analysis 


Region 

Facility 

type 

Facility  name 

Median  daily  laboratory  test 
orders 

Ft.  Benning 

Hospitals 

MARTIN  ACH 

114 

Clinics 

RECEPTION  STA.  TMC-FT.  BENNING 

0 

TMC-l-FT.  BENNING 

1 

TMC-2-FT.  BENNING 

0 

TMC-5-FT.  BENNING 

2 

TMC-7-FT.  BENNING 

2 

WINDER  FPC 

14 

Hawaii 

Hospitals 

TRIPLER  AMC 

94 

Clinics 

15th  MEDICAL  GROUP 

10 

BMC  MCAS  KANEOHE  BAY 

7 

NBHC  MCB  CAMP  H.M.  SMITH 

0 

NBHC  NAVCAMS  EASTPAC 

0 

NHC PEARL  HARBOR 

27 

SCHOFIELD  BARRACKS  AHC 

27 

TMC-1 -SCHOFIELD  25th 

5 

NCA 

Hospitals 

89th  MEDICAL  GROUP 

61 

NNMC  BETHESDA 

79 

WALTER  REED  ARMY  MEDICAL  CENTER 

59 

Clinics 

1 1TH  MEDICAL  GROUP 

10 
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Region 

Facility 

type 

Facility  name 

Median  daily  laboratory  test 
orders 

ANDREW  RADER  AHC 

11 

BMC  WILLOW  GROVE 

1 

DEWITT  ACH 

52 

DILORENZO  TRICARE  HEALTH  CLINICS 

7 

FAMILY  HEALTH  CENTER  FAIRFAX 

24 

FAMILY  HEALTH  CENTER  WOODBRIDGE 

44 

KIMBROUGH  AMBULATORY  CARE  CENTER 

30 

KIRK  AHC 

10 

NBHC  ANDREWS  AFB 

0 

NBHC  DAHLGREN 

0 

NBHC  INDIAN  HEAD 

1 

NHC  ANNAPOLIS 

7 

NHC  PATUXENT  RIVER 

11 

NHC  QUANTICO 

26 

Pensacola 

Hospitals 

NH  PENSACOLA 

56 

Clinics 

NBHC  MILTON  WHITING  FIELD 

3 

NBHC  NAS  PENSACOLA 

3 

NBHC  NATTC  PENSACOLA 

8 

NBHC  NTTC  PENSACOLA 

2 

San  Diego 

Hospitals 

NH  CAMP  PENDLETON 

46 

NMC  SAN  DIEGO 

112 

Clinics 

BMC  CAMP  DELMAR  MCB 

0 

BMC  CORCEN  MCB 

0 

BMC  EDSON  RANGE  ANNEX 

3 

BMC  MCAS  MIRAMAR 

13 

BMC  MCB  CAMP  PENDLETON 

0 

BMC  SAN  ONOFRE  MCB 

1 

NBHC  CORONADO 

0 

NBHC  EL  CENTRO 

0 

NBHC  MCRD  SAN  DIEGO 

8 

NBHC  NAS  NORTH  ISLAND 

5 

NBHC  NAVSTA  SAN  DIEGO 

5 

NBHC  NTC  SAN  DIEGO 

12 

TRICARE  OUTP AT IENT -CHULA  VISTA 

19 

TRICARE  OUTPATIENT-CLAIRMONT 

17 

TRICARE  OUTPATIENT -OCEANSIDE 

4 

Wright- 

Hospitals 

74th  MEDICAL  GROUP 

79 

Patterson 

All  hospitals 

67 

All  clinics 

4 

All  MTFs 

7 

Daily  time  series  were  constructed  for  each  laboratory  test  and  outpatient  visit 
syndrome  by  region.  The  time  series  covers  the  two-year  period  from  1  November  2002 
through  3 1  October  2004.  For  each  day  of  the  study  period,  the  number  of  outpatient 
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visits  for  each  syndrome  and  the  number  of  each  of  the  7 1  laboratory  tests  were  counted 
for  each  region. 

Clinics  typically  order  a  small  number  of  laboratory  tests  every  day.  The  median 
number  of  laboratory  tests  ordered,  across  selected  clinics  for  two  years,  is  only  four  per 
day.  Time  series  correlations  among  series  with  such  small  counts  are  quite  low,  and 
may  reflect  the  sparseness  of  the  data  more  than  any  lack  of  association  between 
outpatient  visits  and  laboratory  test  orders.  However,  combining  clinic  data  with  data 
from  nearby  hospitals  may  be  a  good  way  to  evaluate  regional  ordering  patterns.  All 
correlations  for  this  specific  aim  will  be  performed  at  the  regional  level. 

Both  hospitals  and  clinics  tend  to  order  more  tests  on  weekdays  than  on 
weekends.  This  pattern  has  been  observed  in  multiple  syndromic  surveillance  data 
sources  (e.g.  Lazarus,  2001  [ref])  and  is  illustrated  in  Figure  4.1.  In  the  selected  DoD 
hospitals,  laboratory  tests  are  three  times  more  likely  to  be  ordered  on  weekdays  than  on 
weekends,  and  in  clinics  the  weekday/weekend  ratio  is  even  higher.  For  common 
laboratory  cultures  the  weekday/weekend  ratio  ranges  from  1 : 1  to  one  for  blood  tests  in 
hospitals  to  more  than  4:1  for  throat,  wound,  and  stool  cultures  in  clinics. 


Figure  4.1:  Day-of-the-week  effect  in  laboratory  test  orders 
Time  series  correlations  among  series  with  similar  day-of-the-week  effects  are 
likely  to  be  high  because  some  of  the  correlation  may  be  due  to  the  day-of-the-week 
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effect  rather  than  to  actual  changes  in  disease  patterns.  For  example,  in  the  National 
Capital  area  (NCA),  the  time  series  correlation  between  stool  cultures  and  respiratory 
diagnoses  is  0.57  when  calculated  on  the  raw  data  and  less  than  0.01  when  the  day-of-the 
week  effect  is  removed.  Similarly,  the  correlation  between  throat  cultures  and  GI 
diagnoses  is  0.80  when  calculated  on  the  raw  data  and  0.47  when  the  day-of-the-week 
effect  is  removed.  (Day-of-the-week  effects  were  removed  by  calculating  seven-day 
moving  averages,  as  described  below.) 

The  time  series  should  be  corrected  for  day-of-the-week  effects  in  order  to  focus 
specifically  on  changes  due  to  disease  patterns  rather  than  changes  due  to  weekly  cycles. 
To  remove  the  weekly  cycle  we  calculated  7-day  centered  moving  averages.  The  7-day 
centered  moving  average  for  a  particular  day  is  defined  as  the  arithmetic  mean  of  the 
counts  on  the  current  day,  the  three  previous  days,  and  the  three  following  days. 

It  is  possible  for  two  related  time  series  to  have  a  low  correlation  if  one  of  the 
series  lags  behind  the  other.  For  example,  if  patients  diagnosed  with  GI  illness  are 
typically  asked  to  return  for  a  second  visit  three  days  after  the  first,  and  only  give  a  stool 
sample  after  the  second  visit,  then  the  time  series  for  GI  syndrome  and  stool  culture  will 
be  correlated  with  a  three-day  lag.  This  relationship  would  not  be  observed  if 
correlations  were  only  calculated  between  syndrome  counts  and  laboratory  tests  on  the 
same  day.  We  examined  correlations  between  syndrome/laboratory  test  pairs  for  the 
same  day,  after  lagging  syndrome  counts  by  up  to  six  days,  and  after  lagging  laboratory 
test  counts  up  to  six  days.  The  highest  of  these  13  correlations  was  selected  as  best 
describing  the  relationship  between  the  syndrome/laboratory  test  pair. 
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In  25  percent  of  the  syndrome/laboratory  test  pairs,  the  laboratory  test  time  series 
did  not  lead  or  lag  behind  the  syndrome  time  series.  The  syndrome  series  lagged  behind 
the  laboratory  test  series  37  percent  of  the  time,  and  the  laboratory  test  series  lagged 
behind  the  syndrome  series  37  percent  of  the  time.  These  results  suggest  that  there  is  no 
overall  advantage  to  one  data  source  or  the  other  with  respect  to  timeliness.  However,  for 
particular  syndromes  and  regions,  one  data  source  may  provide  an  earlier  indication  of 
changes  in  disease  patterns. 

Eighteen  percent  of  the  syndrome/laboratory  test  pairs  in  the  six  selected  regions 
had  time  series  correlations  greater  than  0.2.  Associations  are  listed  in  Table  4.4.  These 
correlations  were  further  compared  with  the  CDC  expert  panel’s  laboratory  test/syndrome 
associations.  Results  are  described  below.Table  4.4:  Tests  associated  with  each 
syndrome,  based  on  criterion  of  correlation  >0.2  in  at  least  one  region 
Botulism-like  illness  syndrome 

The  time  series  correlations  generally  support  the  CDC  expert  panel’s  suggestion 
that  no  laboratory  tests  are  expected  to  be  associated  with  the  botulism-like  illness 
syndrome.  All  correlations  with  this  syndrome  were  less  than  0.5.  Among  the  tests  most 
correlated  with  this  syndrome  were  urine  cultures  (r  =  0.45  in  San  Diego  and  r  =  0.27  in 
Hawaii).  Urine  cultures  are  the  most  commonly  ordered  test  overall,  so  this  association 
may  simply  reflect  frequency  of  outpatient  visits.  Some  tests  for  sexually  transmitted 
infections  may  be  associated  with  botulism-like  illness  visits;  the  correlation  with 
Chlamydia  cultures  at  Ft.  Benning  was  0.28  and  the  correlation  with  other  STIs  in  the 


NCA  was  0.48. 
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Table  4.4:  Tests  associated  with  each  syndrome,  based  on  criterion  of  correlation  >0.2 


in  at  least  one  region 


Botlike 

Fever 

GI 

Hemrill 

OTHER  STD 

THR  CULT 

THR  CULT 

UA  CULT 

UA  CULT 

VIRAL  CULT 

GC  CULT  SMEAR 

BLD  PARA 

O  P 

GROUP  A  STREP 

UA  CULT 

FUNGUS  GENITAL 

RESP  CULT 

GC  CULT  SMEAR 

GROUP  A  STREP 

OTHER  STD 

WND  CULT 

SPUTUM  CULT 

LEISHMANIASIS 

GENITAL  CULTURE 

GROUP  B  STREP 

CHLAMYDIA  CULT 

INFLUENZA 

CHLAMYDIA  CULT 

STOOL  CULT 

BLD  CULT 

FOB 

CSF  CULTURE 

FUNGUS  CSF 

AER  BLD  CULT 

GENITAL  CULTURE 

GC  CULT  SMEAR 

MISC  CULTURE 

STOOL  CULT 

OTHER  STD 

VIRAL  CULT 

FUNGUS  OTHER 

INFLUENZA 

H  PYLORI  CULT 

GROUP  B  STREP 

GENITAL  CULTURE 

GRAM  STAIN 

STOOL  CULT 

NASAL  CULT 

FUNGUS  RESP 

AFB 

FUNGUS  OTHER 

FUNGUS  CSF 

CHLAMYDIA  CULT 

EYE  CULT 

FUNGUS  GENITAL 

O  P 

BLD  PARA 

FUNGUS  CSF 

FECAL  WBC 

ANAER  CULT 

FOB 

FOB 

RSV 

FUNGUS  OTHER 

GC  CULT  SMEAR 

RESP  CULT 

O  P 

MRSA 

AFB 

UA  CULT 

NASAL  CULT 

FOB 

SPUTUM  CULT 

BLD  PARA 

BRUCELLA  CULT 

ANAER  BLD  CULT 

PINWORM 

OTHER 

RESP  CULT 

STOOL  CULT 

THR  CULT 

RSV 

AFB 

THR  CULT 

GRAM  STAIN 

GENITAL  CULTURE 

OTHER  GI 

ACINETOBACTER 

FUN  GU  S _ GENIT  AL 

VARICELLA 

WND  CULT 

SPUTUM  CULT 

ANAER  CULT 
VARICELLA 

HERPES 

EYE  CULT 

PAPSMEAR 

LEISHMANIASIS 

GRAM  STAIN 

OTHER 

AEROBIC  CULT 

BF  CULT 

RESP  CULT 

SPUTUM  CULT 
BRUCELLA  CULT 

Neuro 

Rash 

Resp 

ShkComa 

FUNGUS  GENITAL 

GROUP  B  STREP 

THR  CULT 

GROUP  B  STREP 

OTHER  STD 

UA  CULT 

GROUP  A  STREP 

MRSA 

UA  CULT 

ANAER  CULT 

VIRAL  CULT 

ANAER  CULT 

EYE  CULT 

0  P 

GC  CULT  SMEAR 

O  P 

H  PYLORI  CULT 

GC  CULT  SMEAR 

INFLUENZA 

FUNGUS  GENITAL 

FUNGUS  OTHER 

OTHER  STD 

BLD  CULT 

LEGIONELLA  CUL 

GENITAL  CULTURE 

WND  CULT 

RESP  CULT 

GRAM  STAIN 

OTHER 

FUNGUS  OTHER 

AFB 

BF  CULT 

GROUP  B  STREP 

MRSA 

RSV 

RESP  CULT 

WND  CULT 

BF  CULT 

UA  CULT 

FUNGUS  OTHER 

CHLAMYDIA  CULT 

AFB 

FOB 

WND  CULT 

ACINETOBACTER 

GRAM  STAIN 

SPUTUM  CULT 

SPUTUM  CULT 

BLD  PARA 

FUNGUS  RESP 

CHLAMYDIA  CULT 

UA  CULT 

CSF  CULTURE 

THR  CULT 

GENITAL  CULTURE 

OTHER  GI 

MISC  CULTURE 
AEROBIC  CULT 

AER  BLD  CULT 
PINWORM 

BRUCELLA  CULT 

OTHER 

EYE  CULT 

GRAM  STAIN 
BORDETELLA  CUL 
OTHER  GI 

FUNGUS  CSF 

AER  BLD  CULT 

NASAL  CULT 

CSF  CULTURE 

STOOL  CULT 

HERPES 

GC  CULT  SMEAR 

BLD  CULT 

BLD  PARA 
CHLAMYDIA  CULT 
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Fever  syndrome 

The  CDC  expert  panel  associated  blood,  viral  and  MRSA  cultures  and  Gram 
stains  with  the  fever  syndrome.  This  study  found  no  association  between  fever  visits  and 
MRSA  tests,  and  a  slight  association  between  fever  visits  and  Gram  stains  in  one  region 
(r  =  0.33  in  San  Diego).  Blood  cultures  showed  the  highest  association  with  fever 
diagnoses,  with  correlations  greater  than  0.2  in  all  regions.  Correlations  range  from  0.27 
in  Hawaii  to  0.44  in  the  NCA.  Viral  cultures  were  strongly  associated  with  fevers  in  two 
of  the  six  regions  (r  =  0.61  in  Hawaii,  r  =  0.72  in  the  NCA).  San  Diego  did  not  report  any 
viral  cultures  during  this  period. 

Throat  cultures  are  highly  correlated  with  the  fever  syndrome.  Correlations  were 
greater  than  0.6  in  all  regions  except  San  Diego,  where  the  correlation  was  0.23.  Group 
A  strep  was  highly  correlated  with  fevers  in  the  NCA  (r  =  0.67)  and  at  Wright-Patterson 
(r  =  0.62).  These  findings  suggest  considerable  overlap  between  the  fever  and  respiratory 
syndromes. 

Gastrointestinal  syndrome 

Time  series  correlations  with  the  GI  syndrome  were  uniformly  low.  The  strongest 
correlation  was  with  throat  cultures  in  Hawaii  (r  =  0.63).  Stool  cultures  were  moderately 
correlated  with  GI  illness  in  all  regions,  with  correlations  ranging  from  0.25  in  San  Diego 
to  0.39  in  the  NCA.  Other  GI  tests  identified  by  the  CDC  expert  panel  that  had 
correlations  of  at  least  0.2  with  GI  illness  in  at  least  one  region  are  fecal  white  blood 
cells,  fecal  occult  blood,  H.  pylori,  and  ova  and  parasites.  C.  difficile,  E.  coli,  fecal 
reducing  substances,  Giardia/crypto,  rotavirus,  vibrio,  yersinia,  and 
salmonella/shigella/campylobacter  tests  were  associated  with  GI  illness  by  the  CDC 
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expert  panel  but  were  not  correlated  with  GI  syndrome  diagnoses  in  this  analysis.  All  of 
these  tests  are  ordered  infrequently,  so  the  basis  for  estimating  these  correlations  is  weak. 
Finally,  the  time  series  analysis  identified  an  association  between  GI  illness  and  Group  A 
strep  that  was  not  identified  by  the  CDC  panel  (r  =  0.5 1  in  the  NCA,  and  r  =  0.53  at 
Wright-Patterson  AFB). 

Hemorrhagic  illness  syndrome 

The  time  series  correlations  generally  support  the  CDC  expert  panel’s  suggestion 
that  no  laboratory  tests  are  expected  to  be  associated  with  the  hemorrhagic  illness 
syndrome.  All  correlations  with  this  syndrome  were  less  than  0.6.  Among  the  tests  most 
correlated  with  this  syndrome  were  urine  cultures  (r  =  0.40  in  Hawaii  and  r  =  0.56  in  the 
NCA).  As  noted  for  the  botulism-like  illness  syndrome,  urine  cultures  are  the  most 
commonly  ordered  test  overall,  so  this  association  may  simply  reflect  increased 
frequency  of  visits  to  healthcare  providers  who  then  order  standard  tests.  Other  moderate 
correlations  with  hemorrhagic  illness  include  blood  parasite  tests  (r  =  0.47  at  Ft. 
Benning),  genital  fungus  tests  (r  =  0.46  in  Hawaii)  and  other  STIs  (r  =  0.41  in  the  NCA). 
Neurological  illness  syndrome 

Neurological  illness  is  a  rare  syndrome,  with  fewer  than  5,000  diagnoses  in  the 
six  selected  regions  over  a  two-year  period  (fever,  by  comparison,  was  diagnosed  nearly 
142,000  times).  None  of  the  correlations  with  neurological  illness  exceeded  0.5,  perhaps 
because  of  the  infrequent  diagnosis.  The  CDC  expert  panel  identified  cerebrospinal  fluid 
(CSF)  cultures  as  associated  with  neurological  illness,  and  this  is  somewhat  confirmed  in 
the  time  series  analysis.  Three  of  the  six  regions  showed  correlations  between 
neurological  illness  and  CSF  culture  exceeding  2.0  (r  =  0.28  in  Hawaii,  r  =  0.23  in 
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Pensacola,  r  =  0.30  in  San  Diego).  There  were  several  other  tests  with  correlations 
between  0.2  and  0.5,  but  no  consistent  patterns. 

Rash  syndrome 

The  CDC  expert  panel  associated  skin,  viral,  and  wound  cultures,  and  herpes  and 
varicella  tests,  with  rash  syndrome.  Of  these  associations,  only  wound  cultures  is 
confirmed  by  the  time  series  analysis,  and  even  then  the  association  is  only  observed  in 
the  NCA  (r  =  0.35)  and  San  Diego  (r  =  0.34).  Other  notable  correlations  were  with 
MRSA  (r  =  0.30  in  the  NCA),  fungus  tests  (r  =  0.31  in  the  NCA  and  r  =  0.29  in  San 
Diego),  group  B  strep  (r  =  0.53  in  the  NCA  and  r  =  0.44  in  San  Diego),  and  anaerobic 
cultures  (r  =  0.45  in  the  NCA  and  r  =  0.3 1  in  San  Diego). 

Respiratory  syndrome 

Throat  cultures  were  correlated  with  respiratory  syndrome  diagnoses  in  all  six 
regions.  Correlations  ranged  from  0.35  in  San  Diego  to  0.85  in  Pensacola,  and  exceeded 
0.8  in  four  out  of  five  regions.  In  San  Diego,  the  highest  correlation  with  respiratory 
illness  was  for  respiratory  cultures  (r  =  0.43)  rather  than  throat  cultures  (r  =  0.35).  It  is 
likely  that  this  represents  a  difference  in  tenninology  among  regions  rather  than  a 
difference  in  medical  practice. 

Tests  for  group  A  strep  were  strongly  associated  with  respiratory  illness  in  two 
regions  (r  =  0.77  in  the  NCA  and  r  =  0.72  at  Wright-Patterson),  but  this  correlation  was 
less  than  0.2  in  the  other  four  regions.  Similarly,  viral  cultures  were  strongly  associated 
with  respiratory  illness  in  the  NCA  (r  =  0.67)  but  not  in  the  other  regions  (r  =  0.24  in 
Hawaii,  and  r  <  0.2  in  the  other  four  regions). 
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The  CDC  expert  panel  associated  several  more  specific  tests  with  the  respiratory 
syndrome.  Four  of  these  associations  (Bordetella  pertussis,  Chlamydia,  influenza  and 
RSV)  were  identified  for  at  least  one  region  in  the  time  series  analysis,  while  the  rest 
(AFB,  Brucella,  respiratory  fungus,  Gram  stains,  Legionella,  and  mycoplasma)  were  not. 

Blood  cultures  were  associated  with  respiratory  illness  in  the  time  series  analysis,  but  not 
by  the  CDC  expert  panel;  correlations  ranging  from  0.24  to  0.44  were  observed  in  all  six 
regions. 

Shock/coma  syndrome 

No  correlations  with  the  shock/coma  syndrome  exceeded  0.6.  This  is  a  rare 
syndrome,  with  only  2,105  diagnoses  in  the  five  regions  during  the  two-year  period.  One 
interesting  finding  for  this  syndrome,  however,  is  that  the  correlation  with  ova/parasite 
tests  exceeded  0.2  for  two  regions  (r  =  0.22  in  Hawaii,  r  =  0.36  in  the  NCA).  This  is 
consistent  with  the  finding  that  ova  and  parasite  exams  are  three  times  more  likely  to  be 
ordered  in  conjunction  with  a  shock/coma  syndrome  diagnosis  than  would  be  expected  by 
chance. 

In  summary,  the  associations  identified  under  this  criterion  are  similar  to  the 
associations  proposed  by  the  CDC  expert  panel,  but  very  few  correlations  exceeded  0.6. 

The  strongest  correlations,  not  surprisingly,  are  with  respiratory  illness,  a  high- volume 
syndrome  with  a  strong  seasonal  pattern.  A  few  associations  that  were  not  suggested  by 
the  CDC  expert  panel,  such  as  MRS  A  and  rash,  blood  cultures  and  respiratory  illness, 
and  group  A  strep  and  fever,  should  be  considered  further  for  surveillance. 
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Criterion  #3:  Outbreak  peaks 

The  final  criterion  for  associating  laboratory  tests  with  syndromes  is  signal-to- 
noise  ratio  during  outbreaks.  A  laboratory  test  is  considered  to  be  associated  with  a 
disease  syndrome  if  orders  exceed  the  usual  volume  by  at  least  two  standard  deviations 
during  an  outbreak  of  the  disease  syndrome.  The  signal-to-noise  ratio  (SNR)  is  defined 
as 

(peak  count-mean  count)  /standard  deviation  (SD)  of  counts 
where  the  mean  and  standard  deviation  of  counts  are  calculated  during  a  baseline  period 
preceding  the  outbreak,  and  the  peak  count  is  defined  as  the  maximum  daily  count 
observed  during  the  defined  outbreak  period.  Seven-day  moving  averages  are  used  rather 
than  raw  counts  in  order  to  eliminate  the  day-of-the-week  effect.  This  criterion  was  used 
by  researchers  from  WRAIR  and  the  Johns  Hopkins  University  Applied  Physics  Lab 
(JHU/APL)  in  their  evaluation  of  the  BioNet  disease  surveillance  system  in  San  Diego, 
California  [64]. 

Signal-to-noise  ratios  are  calculated  during  verified  outbreaks.  Most  such 
outbreaks  are  for  respiratory  and  gastrointestinal  illnesses.  The  evaluation  of  the  BioNet 
program  focused  on  three  outbreaks  in  San  Diego:  the  October  2003  wildfires,  the 
influenza  season  of  2003-2004,  and  an  outbreak  of  suspected  norovirus  at  the  Marine 
Corps  Recruit  Depot  in  early  2004.  This  study  will  focus  on  the  same  outbreaks  for 
comparability.  In  addition  we  will  consider  the  2003-2004  influenza  season  in  the  NCA, 
in  order  to  explore  regional  differences  in  test  ordering  patterns  during  outbreaks. 

San  Diego  Wildfires 
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Johnson  et  al.  describe  the  wildfires  in  their  article,  “Leveraging  Syndromic 
Surveillance  During  the  San  Diego  Wildfires,  2003”  [65]: 

“On  October  25,  2003,  one  of  the  largest  fires  in  California  history 
began  in  San  Diego  County.  Over  a  period  of  three  days,  the  air  quality 
deteriorated  to  unhealthy  and  hazardous  levels,  prompting  school 
cancellations  and  the  general  public  to  stay  at  home.” 

During  this  event,  the  authors  monitored  ambulance  calls,  emergency  department 
visits,  and  over-the-counter  medication  sales,  and  observed  increases  in  respiratory 
indicators  including  asthma-related  emergency  department  visits  and  local  sales  of 
respiratory  medications.  The  BioNet  evaluation  team  found  in  addition  that  outpatient 
visits  to  MTFs  for  asthma-related  conditions  increased  during  this  event. 

Examination  of  SNRs  for  each  laboratory  test  during  October  2003  identified  only 
one  laboratory  test,  tissue  cultures,  with  a  SNR  greater  than  two  during  the  wildfire  event. 
However,  examination  of  a  time  series  plot  of  tissue  cultures  suggests  that  this  peak  is 
part  of  an  increase  in  tissue  cultures  beginning  in  late  September,  so  it  is  likely  unrelated 
to  the  wildfires.  Even  during  this  peak,  there  were  generally  no  more  than  one  or  two 
tissue  cultures  ordered  per  day  in  the  San  Diego  region.  The  laboratory  test  data  may  not 
be  the  best  data  source  for  tracking  the  health  consequences  of  this  event.  This  is  not 
surprising  since  microbiology  lab  tests  are  expected  to  be  ordered  more  often  in  the 
presence  of  an  infectious  disease,  and  the  wildfires  represent  exposure  to  irritants  in  the 
environment. 


Annual  influenza 
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Several  laboratory  tests  peaked  strongly  during  the  2003-2004  influenza  season  in 
San  Diego  (Figure  4.2a).  Throat  cultures  showed  the  strongest  association,  with  a  SNR 
of  7.60  (compared  with  a  SNR  of  5.03  for  respiratory  diagnoses),  and  a  peak  eight  days 
before  the  peak  for  respiratory  diagnoses.  Throat  cultures  also  show  several  peaks  during 
the  summer  and  fall  of  2003;  it  is  unclear  whether  these  represent  true  outbreaks  or 
random  variability. 

The  SNR  for  respiratory  cultures  was  6.57  during  influenza  season,  but 
respiratory  cultures  lagged  behind  other  indicators  of  influenza,  with  a  peak  in  early 
January.  Blood  cultures  showed  a  peak  SNR  of  3.3 1  but  did  not  peak  until  after 
respiratory  visits.  Gram  stains  showed  a  peak  SNR  of  6.2  one  day  before  the  peak  for 
respiratory  diagnoses,  but  also  show  several  additional  peaks  in  the  fall  so  may  not  be  the 
most  specific  indicator  of  the  influenza  season  (data  not  shown). 

Throat,  respiratory  and  blood  cultures  also  peaked  in  the  NCA  during  the  2003- 
2004  influenza  season  (figure  4.2b).  Throat  cultures  track  very  closely  with  respiratory 
visits  and  show  a  similar  peak  SNR  (3.76,  compared  with  4.38  for  respiratory  visits). 
Respiratory  cultures  increased  slightly  during  influenza  season,  with  a  short  peak  in  early 
December  and  a  max  SNR  of  2.73.  Blood  cultures  showed  a  higher  peak  (SNR  =  8.26). 
The  strongest  signal  in  the  NCA  was  for  viral  cultures,  which  were  not  ordered  at  all  in 
San  Diego.  The  SNR  was  27. 10,.  The  SNR  for  viral  cultures  first  exceeded  2.0  a  day 
earlier  than  the  SNR  for  respiratory  visits,  and  the  SNR  for  viral  cultures  stayed  elevated 
throughout  influenza  season.  The  significance  of  viral  cultures  in  the  NCA,  when  no 
tests  were  reported  by  this  name  in  San  Diego,  illustrates  the  importance  of  regional 
differences  in  test  ordering  patterns. 
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Gastrointestinal  outbreak  at  MCRD  San  Diego 

An  outbreak  of  suspected  norovirus  occurred  in  the  San  Diego  region  in  January 
2004.  Outpatient  visits  for  gastrointestinal  illness  were  elevated  for  two  months,  with  a 
SNR  of  18.45  on  January  15th.  Several  laboratory  tests  also  increased  during  this 
outbreak:  fecal  white  blood  cells  peaked  on  January  17th,  with  a  SNR  of  3.60,  and  stool 
cultures  peaked  on  January  18th  with  a  SNR  of  3.69.  Both  of  these  tests  showed  lower 
SNRs  and  later  peaks  than  outpatient  visits  for  gastrointestinal  illness,  so  it  is  not  clear 
whether  laboratory  tests  can  improve  surveillance  for  this  syndrome.  The  association  of 
these  two  tests  with  the  outbreak  is  promising,  however,  since  laboratory  data  may  be 
available  for  surveillance  in  a  more  timely  fashion  than  outpatient  visit  data. 

Other  laboratory  tests  associated  with  this  outbreak  include  fecal  occult  blood 
tests  (SNR  3.65,  peak  February  1)  and  pinworms  (SNR  4.92,  peak  January  26).  Both  of 
these  tests  show  a  markedly  later  outbreak  signal  than  outpatient  visits,  so  they  may  not 
be  useful  for  detecting  outbreaks.  However,  they  may  still  be  useful  for  monitoring 
outbreaks.  Finally,  genital  cultures  (SNR  4.04,  peak  January  17)  and  Gram  stains  (SNR 
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5.61,  peak  January  10)  increased  during  this  outbreak. 


San  Diego  Influenza,  2003-2004 


National  Capitol  Area  Influenza,  2003-2004 


San  Diego  Gastrointestinal  Outbreak 


Figure  4.2:  Patterns  of  laboratory  test  ordering  during  known  outbreaks 


In  summary,  this  criterion  identified  a  subset  of  laboratory  tests  that  were 
associated  with  respiratory  and  gastrointestinal  illness  under  the  first  two  criteria,  that 
also  increased  during  influenza  and  suspected  norovirus  outbreaks,  respectively. 
Different  laboratory  tests  were  associated  with  influenza  in  different  regions, 
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underscoring  the  importance  of  understanding  regional  health  care  variations  when  using 
electronic  records  for  disease  surveillance. 

Discussion  and  recommendations 

The  goal  of  this  analysis  is  to  identify  laboratory  tests  that  might  be  useful  for 
disease  surveillance.  Based  on  our  results,  we  propose  for  surveillance  the  five  syndrome 
groups  listed  in  table  4.5.  These  groups  correspond  closely  to  those  proposed  by  the 
CDC  expert  panel;  differences  are  discussed  below. 

Fever  syndrome 

Anaerobic  and  aerobic  cultures  are  not  included  in  our  list  but  are  included  on  the 
CDC  list.  The  CDC  list  does  not  distinguish  between  aerobic  and  anaerobic  cultures  of 
blood  and  other  body  fluids.  The  DoD  data  allow  this  distinction,  and  only  blood 
cultures  are  associated  with  fever  in  the  DoD  data.  The  CDC  list  includes  MRSA  tests 
under  the  fever  syndrome,  but  we  found  a  stronger  association  between  MRSA  and  rash. 

Group  A  strep  tests  showed  a  strong  association  with  fever  syndrome  in  our  analysis  but 
we  chose  instead  to  include  these  tests  in  the  respiratory  syndrome  as  proposed  by  the 
CDC. 

Gastrointestinal  syndrome 

All  tests  proposed  by  the  CDC  expert  panel  were  confirmed  by  criterion  #1  and 
included  on  our  list.  The  other  criteria  confirmed  only  fecal  WBC,  fecal  occult  blood, 
ova/parasites  and  stool  cultures,  probably  because  the  other  tests  are  ordered  rarely,  or  are 
ordered  as  part  of  a  generic  stool  culture  rather  than  by  name. 
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Neurological  syndrome 

This  syndrome  includes  only  CSF  cultures  and  CSF  fungus  tests,  as  proposed  by 
the  CDC  panel.  Our  analysis  identified  other  associated  tests,  including  other  fungus, 
Gram  stains,  herpes,  varicella,  and  viral  cultures.  All  of  these  tests  are  associated  with 
other  syndromes  in  addition  to  neurological,  and  may  not  be  specific  to  neurological 
illness. 

Rash  syndrome 

Our  analysis  confirmed  the  associations  identified  by  the  CDC  expert  panel,  along 
with  anaerobic  cultures,  other  fungus,  and  MRSA.  Anaerobic  cultures  were  identified  by 
both  criteria  #1  and  #2,  and  specimens  for  these  cultures  tended  to  be  skin  and  wounds. 
Other  fungus  is  not  a  category  in  the  CDC  laboratory  data.  Specimens  for  this  test  are 
mostly  skin  and  nails. 

Respiratory  syndrome 

Several  tests  identified  by  the  CDC  expert  panel  were  not  associated  with 
respiratory  illness  in  this  analysis:  acid-fast  bacilli  (AFB),  Brucella,  respiratory  fungus, 
Legionella,  and  Mycoplasma.  AFB  may  reflect  chronic  illness,  and  the  others  are  rarely 
ordered.  If  they  were  included  in  this  syndrome  it  is  likely  that  these  rare  tests  would 
disappear  in  the  “noise”  of  throat,  respiratory  and  viral  cultures.  If  the  rare  tests  are  of 
specific  interest  they  should  be  monitored  separately  from  respiratory  syndrome. 
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Table  4.5:  Proposed  syndrome  grouping  for  microbiology  laboratory  tests 


Botlike 

Fever 

GI 

Hemr_ill 

NONE 

AER  BLD  CULT 
ANAER  BLD 
CULT 

BLD  CULT 

BLD  PARA 

GRAM  STAIN 
VIRAL  CULT 

C  DIFFICILE 

E  COLI  0157:H7 
FECAL  RS 

FECAL  WBC 

FOB 

GIARDIA/CRYPTO 

H  PYLORI  CULT 
O&P 

OTHER  GI 
ROTAVIRUS 

SSC 

STOOL  CULT 

VIBRIO 

YERSINIA  CULT 

NONE 

Neuro 

Rash 

Resp 

Shk_Coma 

CSF 

CULTURE 

FUNGUS, 

CSF 

ANAER  CULT 
FUNGUS, 

OTHER 

HERPES 

MRSA 

VARICELLA 
WND  CULT 

BORDETELLA 

CULT 

CHLAMYDIA  CULT 
GROUP  A  STREP 
INFLUENZA 

RESP  CULTURE 

RSV 

THR  CULT 

VIRAL  CULT 

NONE 

Syndrome  associations 

Once  the  final  syndrome  definitions  were  developed,  criteria  #1  and  #2  were 
revisited  to  confirm  that  once  tests  were  combined  into  a  syndrome,  tests  associated  with 
a  syndrome  were  more  likely  to  be  ordered  during  visits  for  the  syndrome,  and  daily 
counts  of  all  tests  in  a  syndrome  are  correlated  over  time  with  outpatient  visits  for  the 
syndrome.  Table  4.6  shows  that  observed/expected  ratios  and  correlations  were  generally 
strong  between  laboratory  test-based  syndromes  and  outpatient  visit-based  syndromes. 
Observed/expected  ratios  were  all  greater  than  3.0  and  correlations  exceeded  2.0  for  three 
of  five  syndromes.  The  neurological  syndrome  had  a  relatively  weak  time  series 


70 


correlation  between  the  two  data  sources,  probably  because  daily  counts  are  so  low,  but 
had  the  highest  observed/expected  ratio,  because  neurological  tests  are  rarely  ordered  in 
the  absence  of  a  diagnosis  of  neurological  syndrome.  The  rash  syndrome  showed  the 
weakest  overall  association  based  on  both  criteria.  This  may  be  driven  by  the  wound 
cultures,  which  are  as  likely  to  be  associated  with  a  fever  diagnosis  as  with  a  rash.  The 
definition  of  the  rash  syndrome  could  be  revisited  in  the  future  depending  on  what 
patterns  are  seen  in  prospective  surveillance. 

Table  4. 6:  Association  between  laboratory  and  ICD-9  syndromes 


Observed/Expected 

ratio 

Median 

correlation 

Fever 

16.42 

0.375 

GI 

14.91 

0.290 

Neuro 

468.90 

0.179 

Rash 

3.64 

0.146 

Resp 

7.28 

0.815 

Further  recommendations 

In  addition  to  identifying  syndrome  groups,  this  analysis  identified  several  other 
issues  important  to  using  laboratory  tests  for  surveillance: 

•  Laboratory  tests  are  currently  archived  only  after  the  test  result  is  certified,  even 
though  test  ordering  information  is  entered  electronically  at  the  time  of  order.  In 
order  for  laboratory  test  data  to  provide  timely  indicators  of  disease  patterns,  the 
electronic  data  collection  system  should  be  modified  to  monitor  laboratory  tests  as 
they  are  ordered. 

•  Laboratory  test  names  should  be  standardized  to  facilitate  consistent  monitoring 
across  MTFs  and  over  time.  Standard  laboratory  test  terminology,  such  as 


LOINC  and  SNOMED  coding,  is  used  in  other  systems  and  is  being  evaluated  for 
the  laboratory  module  of  CHCS  II  [66].  DoD  epidemiologists  may  wish  to 
participate  in  the  development  of  the  laboratory  module  of  CHCS  II,  to  ensure 
that  the  data  are  collected  in  such  a  way  that  they  may  ultimately  be  used  for 
surveillance. 

Our  test  name  standardization  method  is  ad  hoc  and  would  need  to  be  updated 
every  time  a  new  spelling  is  observed.  If  routine  surveillance  of  laboratory  test 
orders  is  implemented  before  test  names  are  standardized,  it  would  be  useful  to 
employ  a  text  extraction  algorithm  that  could  “recognize”  alternate  spellings  of 
the  same  test  name.  The  standard  test  list  developed  for  this  project  could  be  used 
as  a  starting  point  to  train  the  text  extraction  algorithm. 

Monitoring  laboratory  tests  grouped  into  broad  syndrome  categories  may  be 
useful  for  detecting  influenza  or  many  gastrointestinal  illness  outbreaks,  but  is 
likely  to  miss  small  to  medium  sized  outbreaks  of  rare  illness.  Laboratory  tests 
for  diseases  of  specific  interest  should  be  monitored  independently  of  the 
syndrome  groups. 

Blood  cultures  are  associated  with  all  five  major  syndromes,  and  do  not  appear  to 
be  indicators  of  any  specific  illness.  Blood  cultures  could  be  monitored 
separately  as  an  indicator  of  the  overall  level  of  severe  infectious  disease  in  a 
population. 

Laboratory  test  monitoring  may  be  most  useful  at  the  hospital  or  regional  level. 
Clinics  tended  to  order  only  about  seven  total  tests  per  day,  and  many  (if  not 
most)  of  these  were  routine  (e.g.  urine  cultures).  Under  regional  surveillance,  if 
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increases  are  noted  in  a  region,  it  might  be  possible  to  drill  down  through  the  data 
and  see  if  they  were  all  ordered  in  the  same  hospital  or  clinic. 

In  summary,  outpatient  visit  data  have  been  shown  to  be  a  useful  data  source  for 
syndromic  surveillance.  Laboratory  tests  are  associated  with  both  outpatient  visit  data 
and  with  disease  outbreaks,  suggesting  that  laboratory  tests  are  a  valid  indicator  of 
disease  patterns  in  the  population.  We  will  further  evaluate  the  proposed  syndrome 
groupings  for  laboratory  tests  in  the  next  chapter. 


CHAPTER  V:  Evaluating  outbreak  detection  performance 


Introduction 

The  second  specific  aim  of  this  study  is  to  evaluate  strategies  for  using  laboratory 
test  orders  for  syndromic  surveillance,  alone  and  in  combination  with  outpatient  visit 
data.  The  evaluation  will  focus  primarily  on  the  respiratory  and  gastrointestinal 
syndromes,  since  several  naturally-occurring  outbreaks  have  been  identified  during  the 
time  frame  of  the  study  for  these  syndromes.  This  chapter  will  describe  the  data, 
outbreaks  and  surveillance  algorithms  used  to  evaluate  the  laboratory  data;  describe  the 
sensitivity,  specificity,  and  timeliness  of  outbreak  detection  based  on  laboratory  test  order 
data  for  respiratory  and  GI  syndromes;  and  present  case  studies  illustrating  surveillance 
for  fever,  neurological,  and  rash  syndromes. 

Description  of  data 

Daily  counts  of  laboratory  test  orders  and  outpatient  visits  are  used  for  this 
analysis.  These  data  sets  are  described  in  detail  in  Chapter  4.  In  brief,  data  from 
November  2002  through  November  2004  were  aggregated  by  data  source,  day,  syndrome 
and  region,  resulting  in  a  file  with  daily  counts  of  outpatient  microbiology  laboratory 
tests  for  each  syndrome  and  region,  and  daily  counts  of  outpatient  visits  to  MTFs  for  each 
syndrome  and  region.  Syndromes  considered  include  fever,  gastrointestinal, 
neurological,  rash,  and  respiratory.  Laboratory  tests  associated  with  each  syndrome  are 
listed  in  chapter  4,  and  outpatient  visit  diagnoses  associated  with  each  syndrome  are 
listed  in  Appendix  3. 
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Outbreaks 

The  evaluation  of  outbreak  detection  performance  is  limited  by  the  availability  of 
only  two  years  of  historical  data  containing  a  limited  number  of  documented  outbreaks. 
Evaluation  using  the  same  data  that  was  used  for  developing  the  syndrome  definitions 
could  result  in  biased  estimates  of  outbreak  detection  performance.  Specifically,  in 
Chapter  4,  we  examined  four  known  outbreaks  in  two  regions  to  identify  laboratory  tests 
that  increased  during  the  outbreaks.  Since  we  defined  the  laboratory  gastrointestinal  and 
respiratory  syndromes  in  part  by  identifying  laboratory  tests  that  increased  during  those 
outbreaks,  by  definition  laboratory  test  orders  for  the  syndromes  must  increase  during 
those  outbreaks.  To  obtain  a  valid  test  of  outbreak  detection,  we  examine  whether 
laboratory  test  orders  for  the  syndromes  increased  during  different  outbreaks  and  in 
different  regions. 

For  this  analysis  we  focus  on  outbreaks  that  were  identified  for  the  Defense 
Advanced  Research  Projects  Agency  (DARPA)  Five  Cities  Evaluation[61,  63],  These 
outbreaks  were  identified  by  an  expert  panel  made  up  of  epidemiologists  from  several 
syndromic  surveillance  research  teams.  The  panel  examined  multiple  syndromic 
surveillance  data  sources,  including  visits  to  outpatient  military  treatment  facilities,  but 
they  did  not  have  access  to  laboratory  test  order  data.  Panel  members  examined  daily 
counts  both  visually  and  using  a  simple  anomaly  detection  algorithm,  and  after  discussion 
came  to  consensus  regarding  dates  and  locations  of  outbreaks.  They  jointly  identified  for 
each  outbreak  the  likely  start  date,  peak  date,  end  date,  and  date  at  which  public  health 
officials  would  have  been  likely  to  respond  (PH  date).  They  were  able  to  confirm  a  few 
of  the  outbreaks  they  identified  using  traditional  surveillance  methods.  Six  of  the 
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outbreaks  identified  by  the  expert  panel  took  place  at  times  and  sites  covered  by  the 
retrospective  laboratory  order  data;  these  outbreaks  are  used  in  this  evaluation.  Dates  and 
locations  of  these  outbreaks  are  listed  in  table  5.1,  and  daily  counts  are  plotted  in  figures 
5.1  and  5.2.  Note  that  laboratory  test  order  data  are  available  beginning  in  November 
2002,  so  only  three  days  of  baseline  data  are  available  for  the  Pensacola  respiratory 
outbreak  that  began  on  November  4,  2002.  After  checking  for  differences  in  baseline 
rates,  we  used  data  from  November  2004  to  “train”  the  detection  algorithms  to  detect  the 
Pensacola  respiratory  outbreak. 


Table  5.1:  DARPA-identified  outbreaks 


Syndrome 

City 

Start  date 

PH  date 

Peak  date 

End  date 

Respiratory 

Charleston 

27- Jan 

3-Feb 

10-Feb 

19 -Mar 

Pensacola 

4-Nov 

10-Dec 

3-Feb 

14- Apr 

Norfolk 

3 -Feb 

18-Feb 

24-Feb 

15-Apr 

Gastrointestinal 

Charleston 

6-Dec 

18-Dec 

17-Dec 

28-Jan 

Norfolk 

1 1-Nov 

9-Dec 

24-Feb 

16- Apr 

Norfolk 

22-Feb 

24-Feb 

24-Feb 

1 1-Mar 

Surveillance  algorithms 

This  analysis  primarily  uses  the  DoD-ESSENCE  algorithm  in  order  to  make  the 
results  of  the  study  most  relevant  for  DoD-ESSENCE  planners  in  the  future.  As 
described  above,  this  algorithm  uses  a  regression  model  (useful  for  large  daily  counts)  but 
switches  to  a  simple  EWMA  algorithm  when  the  regression  model  does  not  fit  well 
(usually  when  daily  counts  are  low).  For  comparison  purposes  we  explored  the  EARS 
Cl  and  C3  algorithms[67]  based  on  CUSUM  methodology,  a  stand-alone  EWMA 
algorithm[56],  and  a  temporal  scanning  algorithm[68].  The  DoD-ESSENCE  algorithm 
was  run  using  SAS  code  obtained  from  DoD-ESSENCE  statistical  programmers,  and  the 
comparison  algorithms  were  run  using  an  Excel  program  provided  by  Howard  Burkom  of 
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the  Johns  Hopkins  University  Applied  Physics  Lab[69].  These  algorithms  are  described 
in  Chapter  2. 

In  general,  as  the  threshold  for  alerting  is  raised,  false  alarms  will  be  less  frequent 
(increased  specificity)  but  smaller  outbreaks  are  more  likely  to  be  missed  (decreased 
sensitivity)  and  outbreaks  will  be  caught  later  in  their  development  (decreased 
timeliness).  To  evaluate  tradeoffs  among  sensitivity,  specificity  and  timeliness,  we  chose 
four  different  background  alerting  rates  (specificities)  and  report  sensitivity  (proportion 
of  outbreaks  detected  before  the  PH  date)  and  timeliness  for  each  background  alerting 
rate.  We  report  background  alerting  rates  rather  than  true  specificities  because  we  do  not 
know  for  sure  whether  an  outbreak  exists  on  any  particular  day.  It  is  impossible  to  obtain 
the  true  specificity  of  the  detection  algorithm  in  this  framework.  The  background  alerting 
rate  is  calculated  as  the  number  of  alerts  (excluding  the  known  outbreak)  divided  by  the 
number  of  days  (excluding  the  known  outbreak).  Timeliness  is  defined  as  the  median 
number  of  days  from  outbreak  start  date  to  first  alert  during  the  outbreak  period. 

Outbreak  periods  that  had  no  alerts  were  counted  as  censored  (time  to  alert  greater  than 
length  of  outbreak)  when  calculating  medians. 

We  also  plot  modified  ROC  (receiver  operating  characteristic)  curves,  with 
number  of  outbreaks  detected  on  the  vertical  axis  and  estimated  background  alerting  rate 
on  the  horizontal  axis  (figures  5.3  and  5.4).  This  allows  comparison  of  different 
algorithms  and/or  data  sources  across  a  range  of  background  alerting  rates. 

Combining  data  sources 

Laboratory  test  order  data  are  not  expected  to  replace  outpatient  visit  data  in 
ESSENCE,  but  to  augment  them.  We  explored  a  simple  method  for  evaluating  the 
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laboratory  test  order  data  in  combination  with  outpatient  visit  data  to  investigate  whether 
combining  the  data  sources  increases  the  signal-to-noise  ratio  in  the  data.  The  DoD- 
ESSENCE  algorithm  yields  /7-values  for  each  data  stream  indicating  the  likelihood  of  a 
daily  count  at  least  as  extreme  as  the  observed  count  if  no  outbreak  is  ongoing,  These 
separate  /7-values  are  combined  to  obtain  a  composite  /7-value  comparing  observed  events 
to  expected  events.  The  methods  of  Edgington  [54]  and  Fisher  [53],  described  in  Chapter 
2,  are  adapted  for  this  analysis.  Neither  is  likely  to  be  implemented  in  ESSENCE;  the 
purpose  of  using  these  methods  for  this  analysis  is  to  quantify  in  a  simple  way  the  added 
value  of  the  laboratory  test  order  data.  In  practice,  when  laboratory  test  order  data  are 
first  incorporated  into  DoD-ESSENCE  it  is  likely  to  be  as  an  informal  check  or 
confirmation  of  the  patterns  observed  in  the  outpatient  visit  data.  Long  range  plans  may 
include  Bayesian  models  for  combining  multiple  data  sources. 

The  methods  of  Edgington  and  Fisher  involve  additive  and  multiplicative 
functions  of  /> values  from  separate  data  streams.  The  authors  describe  a  parametric 
methodology  for  determining  the  significance  of  these  functions.  In  this  analysis,  it 
would  be  inappropriate  to  assume  that  the  laboratory  test  order  data  and  outpatient  visit 
data  are  independent  data  streams,  since  laboratory  tests  are  almost  always  ordered  in 
conjunction  with  an  outpatient  visit.  However,  without  specifying  a  covariance  structure 
for  the  two  data  streams  or  a  distribution  for  the  sums  and  products  of /;- values,  it  is  still 
possible  to  identify  increases  above  baseline.  In  this  analysis  we  use  historical  data  to 
estimate  thresholds  for  alerting  that  will  yield  background  alerting  rates  of  one  per  two 
weeks,  one  per  four  weeks,  one  per  six  weeks,  and  one  per  eight  weeks,  and  examine  the 
sensitivity  and  timeliness  of  the  combined  p-values. 
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Evaluation  results 

Daily  counts  of  respiratory  and  gastrointestinal  laboratory  tests  and  outpatient 
visits  in  outbreak  sites  were  analyzed  using  the  DoD-ESSENCE  algorithm.  For 
respiratory  outbreaks,  DoD-ESSENCE  alerted  before  the  PH  date  in  all  three  outbreaks 
and  both  data  sources  (figure  5.1).  Each  data  source  alerted  during  one  of  the  three 
gastrointestinal  outbreaks,  but  not  during  the  same  outbreak  (figure  5.2).  This  suggests 
the  possibility  that  laboratory  tests  and  outpatient  visits  may  complement  each  other  by 
detecting  different  types  of  outbreaks. 

The  DoD-ESSENCE  algorithm  alerts  when  the  p-value  comparing  observed  to 
expected  counts  exceeds  0.995.  For  direct  comparison  of  data  sources  and  algorithms, 
the  alerting  threshold  was  modified  empirically  so  the  number  of  alerts  excluding  the 
outbreak  period  was  exactly  one  per  two,  four,  six,  or  eight  weeks.  Because  of  the 
modified  thresholds,  the  results  below  do  not  match  the  alerts  plotted  in  figures  5.1  and 
5.2. 

Comparison  of  data  sources  using  the  DoD-ESSENCE  algorithm. 

Laboratory  test  orders  performed  nearly  as  well  as  outpatient  visits  for  detecting 
outbreaks  in  this  data  set  (Table  5.2).  At  a  background  alerting  rate  of  one  alert  every  six 
weeks,  using  laboratory  test  order  data,  the  DoD-ESSENCE  algorithm  was  able  to  detect 
two  out  of  three  respiratory  outbreaks  prior  to  the  public  health  recognition  date,  both  on 
the  first  day  of  the  outbreak.  None  of  the  three  gastrointestinal  outbreaks  were  detected 
prior  to  public  health  recognition  date,  although  all  of  the  three  missed  gastrointestinal 
outbreaks  alerted  before  the  end  of  the  outbreak.  Median  timeliness  for  all  outbreaks 


(including  those  detected  after  the  PH  date)  was  17  days. 
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Table  5.2:  Date  of first  alert,  sensitivity  (number  of  outbreaks  detected)  and  timeliness 
(days  from  outbreak  start  date  to  first  alert),  DoD-ESSENCE  algorithm. 


Outbreak  dates  Date  of  first  alert 


Outbreak 

Start  date 

PH  date 

l/2wks 

1/4  wks 

1/6  wks 

l/8wks 

Laboratory  test  order  data 

Resp- 1 

27-Jan 

3 -Feb 

2 -Feb 

2 -Feb 

ND 

ND 

Resp-2 

4-Nov 

10-Dec 

4-Nov 

4-Nov 

4-Nov 

4-Nov 

Resp-3 

3 -Feb 

18-Feb 

3 -Feb 

3 -Feb 

3 -Feb 

3-Feb 

GI-4 

6-Dec 

18-Dec 

6-Jan 

7- Jan 

7- Jan 

7- Jan 

GI-5 

1 1-Nov 

9-Dec 

1 1-Nov 

1 1-Nov 

13-Jan 

13-Jan 

GI-6 

22-Feb 

24-Feb 

23-Feb 

24-Feb 

24-Feb 

24-Feb 

No.  detected  before  PH  date 

5/6 

4/6 

2/6 

2/6 

Median  timeliness 

0.5 

1 

17 

17 

Outpatient  visit  data 

Resp- 1 

27-Jan 

3 -Feb 

28-Jan 

28-Jan 

1-Feb 

1-Feb 

Resp-2 

4-Nov 

10-Dec 

4-Nov 

4-Nov 

4-Nov 

ND 

Resp-3 

3 -Feb 

18-Feb 

3 -Feb 

3 -Feb 

ND 

ND 

GI-4 

6-Dec 

18-Dec 

9-Dec 

9-Dec 

9-Dec 

9-Dec 

GI-5 

1 1-Nov 

9-Dec 

12-Nov 

7-Dec 

7-Dec 

7-Dec 

GI-6 

22-Feb 

24-Feb 

22-Feb 

ND 

ND 

ND 

No.  detected  before  PH  date 

6/6 

5/6 

4/6 

3/6 

Median  timeliness 

.5 

2 

15 

>26 

ND,  not  detected  (no  alerts  between  outbreak  start  date  and  end  date) 


Using  outpatient  visit  data,  two  of  three  respiratory  outbreaks  and  two  of  three 
gastrointestinal  outbreaks  were  detected  before  the  public  health  recognition  date.  These 
four  outbreaks  were  detected  a  median  of  four  days  after  the  outbreak  start.  Median 
timeliness  for  all  outbreaks  was  15  days.  Figure  5.3  illustrates  the  tradeoff  between 
outbreak  detection  (sensitivity)  and  background  alerting  rate  (proxy  for  specificity).  The 
line  corresponding  to  the  outpatient  visit  data  lies  slightly  above  the  line  for  the 
laboratory  test  order  data  for  background  alerting  rates  below  1  per  10  weeks,  indicating 
that  at  typical  background  alerting  rates,  more  outbreaks  are  likely  to  be  detected  using 
the  outpatient  visit  data. 


80 

A  few  interesting  patterns  were  noticed  with  respect  to  specific  outbreaks.  During 
the  Charleston  gastrointestinal  outbreak  that  began  on  6  December,  laboratory  test  orders 
“bottomed  out”  for  several  days,  suggesting  that  possibly  once  the  outbreak  was 
identified,  clinicians  stopped  ordering  stool  cultures.  If  this  pattern  is  noticed  during 
future  outbreaks,  it  might  be  worth  considering  whether  to  modify  the  alerting  algorithm 
to  check  for  periods  of  unusually  low  counts  as  well  as  elevated  counts.  The  brief, 
explosive  outbreak  in  Norfolk  in  late  February  did  not  alert  at  all  in  the  outpatient  visit 
data,  but  alerted  (albeit  on  the  public  health  recognition  date)  in  the  laboratory  test  order 
data.  So,  while  laboratory  test  order  data  may  not  perform  as  well  overall  as  outpatient 
visit  data,  they  may  be  useful  for  detecting  or  monitoring  some  outbreaks  that  are  not 
recognized  by  existing  syndromic  surveillance. 

Comparison  of  detection  algorithms  using  laboratory  test  order  data. 

Although  the  DoD-ESSENCE  algorithm  performed  reasonably  well  with  the 
laboratory  test  order  data,  a  simple  EWMA  algorithm  may  perform  even  better.  Table 
5.3  shows  that  at  a  background  alerting  rate  of  one  alert  every  six  weeks,  the  EWMA  and 
SCAN  algorithms  detected  more  outbreaks  (four  of  six)  than  the  DoD-ESSENCE 
algorithm  (two  of  six).  The  EWMA  algorithm  detected  algorithms  sooner  than  the  other 
algorithms,  with  a  median  timeliness  of  0.5  days.  No  single  algorithm  dominated  the 
others  with  respect  to  the  sensitivity/background  alerting  rate  tradeoff  (figure  5.2). 
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Table  5.3:  Sensitivity  and  median  timeliness,  selected  detection  algorithms  and 
_ background  alerting  rates,  laboratory  test  order  data. _ 


Number  of  outbreaks  detected  before  public  health  recognition  date  (out  of  six) 

Algorithm  \  alerting  rate 

l/2wks 

1/4  wks 

1/6  wks 

1/8  wks 

ESSENCE 

5 

4 

2 

2 

EARS  Cl 

6 

5 

3 

3 

EARS  C3 

5 

4 

3 

2 

EWMA 

5 

4 

4 

4 

SCAN 

4 

4 

4 

3 

Median  timeliness  (days  from  outbreak  start  to  first  alert) 

Algorithm  \  alerting  rate 

l/2wks 

1/4  wks 

1/6  wks 

1/8  wks 

ESSENCE 

0.5 

1.0 

17.0 

17.0 

EARS  Cl 

2.0 

4.5 

18.0 

18.0 

EARS  C3 

0.0 

1.5 

16.0 

16.0 

EWMA 

0.0 

0.5 

0.5 

1.0 

SCAN 

1.0 

2.0 

4.0 

6.0 

Note:  median  timeliness  calculated  for  all  outbreaks,  including  those  not  detected. 

It  is  somewhat  surprising  that  the  EWMA  algorithm  performs  better  than  the 


DoD-ESSENCE  algorithm  with  laboratory  test  order  data.  The  DoD-ESSENCE 
algorithm  incorporates  an  EWMA  component,  which  is  used  for  alerting  whenever  the  r 
squared  value  from  the  default  regression  model  falls  below  a  threshold.  The  DoD- 
ESSENCE  regression  model  allows  for  weekly  cycles  and  holiday  clinic  closings, 
patterns  that  are  most  pronounced  in  data  sets  with  relatively  large  daily  counts.  Daily 
counts  of  laboratory  test  orders  tend  to  be  fairly  small  in  most  regions.  For  the  three 
regions  and  the  time  frame  discussed  in  this  chapter,  the  median  daily  number  of 
respiratory  laboratory  test  orders  was  21,  and  the  median  daily  number  of  gastrointestinal 
test  orders  was  only  three.  A  priori,  it  seemed  likely  that  the  DoD-ESSENCE  regression 
model  would  not  fit  the  laboratory  data  well,  and  that  most  alerts  would  be  triggered  by 
the  EWMA  component  of  the  algorithm.  This  is  not  the  case,  at  least  for  respiratory 
outbreaks.  The  regression  model  was  used  for  alerting  80  percent  of  the  time.  The 
comparison  of  methods  suggests  that  if  the  EWMA  component  had  been  used  more  often, 
the  detection  performance  might  have  been  better.  (For  gastrointestinal  laboratory  test 
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orders,  the  regression  model  was  used  for  alerting  only  seven  percent  of  the  time,  in 
accordance  with  our  original  hypothesis.)  It  might  be  worthwhile  to  adjust  the  threshold 
for  switching  between  regression  and  EWMA  in  the  DoD-ESSENCE  algorithm  when 
using  laboratory  test  data.  Selecting  an  appropriate  switching  threshold  would  require 
more  data  and  outbreaks  to  evaluate,  and  is  beyond  the  scope  of  this  dissertation. 
Combining  laboratory  test  order  and  outpatient  visit  data. 

/’-values  for  laboratory  test  counts  were  combined  with  /^-values  for  outpatient 
visit  counts  as  described  above.  At  a  background  alerting  rate  of  one  per  six  weeks, 
surveillance  based  on  combined  data  detected  all  three  respiratory  outbreaks  before  the 
PH  date,  and  none  of  the  GI  outbreaks,  although  during  the  brief  Norfolk  outbreak  the 
combined  data  alerted  on  the  PH  date.  Therefore,  sensitivity  of  the  combined  data  (three 
outbreaks  detected)  fell  between  that  of  the  laboratory  data  alone  (two  outbreaks 
detected)  and  the  outpatient  visit  data  alone  (four  outbreaks  detected)  (table  5.4). 
Combining  p-values  improved  the  timeliness  of  outbreak  detection.  Median  time  to  first 
alert  was  more  than  two  weeks  when  each  data  source  was  used  alone,  but  was  only  four 
days  when  the  data  sources  were  combined.  For  three  outbreaks,  the  combined  data 
alerted  on  the  earlier  of  the  laboratory  and  outpatient  visit  alert  dates,  and  for  three 
outbreaks,  the  combined  data  alerted  between  the  laboratory  and  outpatient  visit  alert 
dates.  Combining  data  sources  may  be  a  promising  strategy  for  early  detection  of 
respiratory  and  GI  outbreaks. 
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Table  5.4:  Sensitivity  and  median  timeliness,  data  sources  separate  and  combined. 


Number  of  outbreaks  detected  before  public  health  recognition  date  (out  of  six) 


Method  \  alerting  rate 

l/2wks 

1/4 wks 

1/6  wks 

l/8wks 

Lab  only  (ESSENCE) 

5 

4 

2 

2 

Outpatient  visit  only  (ESSENCE) 

6 

5 

4 

3 

Sum  of  p-values 

4 

3 

3 

3 

Product  of  p-values 

5 

3 

3 

3 

Median  timeliness  (days  from  outbreak  start  to  first  alert) 

Method  \  alerting  rate 

l/2wks 

1/4  wks 

1/6  wks 

l/8wks 

Lab  only  (ESSENCE) 

0.5 

1.0 

17.0 

17.0 

Outpatient  visit  only  (ESSENCE) 

0.5 

2.0 

15.0 

>26 

Sum  of  p-values 

0.5 

3.5 

4.0 

4.0 

Product  of  p-values 

0.0 

3.5 

4.0 

4.0 

Note:  median  timeliness  calculated  for  all  outbreaks,  including  those  not  detected. 


Case  studies:  Fever,  neurological,  and  rash  syndromes 

No  known  outbreaks  have  been  identified  during  the  period  of  this  study  for  fever, 
neurological,  and  rash  syndromes.  Fever  syndrome  is  observed  to  follow  a  seasonal 
pattern  similar  to  the  respiratory  syndrome,  so  it  can  be  evaluated  with  respect  to 
detection  of  seasonal  increases.  Neurological  and  rash  syndromes  are  much  less  common 
than  the  other  syndromes,  so  even  small  increases  in  daily  counts  may  signal  an  alert.  To 
better  describe  the  behavior  of  these  syndromes  in  the  absence  of  known  outbreaks,  we 
present  case  studies. 

Rash  syndrome.  Both  outpatient  visits  and  laboratory  tests  for  rash  syndrome 
alerted  in  summer  2003  in  San  Diego  (figure  5.5).  An  initial  alert  in  laboratory  tests  on 
June  18  was  followed  by  three  consecutive  alerts  on  July  8-10.  These  alerts 
corresponded  to  an  increase  in  wound  cultures.  Three  subsequent  alerts  on  September  3- 
5  corresponded  to  an  increase  in  MRSA  tests.  A  single  alert  in  outpatient  visits  was 
observed  on  June  27,  nine  days  after  the  initial  alert  in  laboratory  test  data.  It  is  possible 
that  these  alerts  represent  a  true  MRSA  outbreak;  a  documented  MRSA  outbreak 
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occurred  in  San  Diego  almost  exactly  one  year  prior  in  military  recruits[70],  If  so,  the 
laboratory  data  provide  an  earlier  and  more  sustained  indication  of  the  outbreak. 

The  national  capital  area  also  experienced  an  increase  in  laboratory  tests  and 
outpatient  visits  for  rash  in  the  summer  of  2003  (figure  5.5).  Outpatient  visits  for  rash 
syndrome  alerted  first  on  June  3,  while  laboratory  tests  for  rash  syndrome  alerted  nearly  a 
week  later  on  June  9.  Laboratory  tests  alerted  again  on  July  2  and  9,  and  outpatient  visits 
alerted  on  three  additional  days  between  August  25  and  September  8.  Laboratory  test 
alerts  corresponded  to  an  increase  in  wound  cultures  with  no  corresponding  or  subsequent 
increase  in  MRSA  tests,  and  it  is  unclear  whether  the  increase  is  related  to  the  San  Diego 
outbreak. 

While  these  increases  do  not  occur  during  documented  outbreaks,  they  suggest 
that  laboratory  test  data  can  complement  outpatient  visit  data  for  detecting  and 
monitoring  increases  in  rash  syndrome.  This  is  encouraging  since  rash  tests  are  usually 
ordered  during  a  visit  that  does  not  result  in  a  diagnosis  of  rash;  only  81  of  more  than 
22,000  laboratory  tests  in  the  rash  syndrome  (0.36%)  have  a  corresponding  rash  diagnosis 
recorded  during  an  outpatient  visit.  Despite  the  lack  of  overlap  between  the  two  data 
sources,  they  both  seem  to  capture  these  potentially  important  increases  in  San  Diego  and 
the  National  Capital  area. 

Neurological. 

This  syndrome  was  the  least  common,  with  average  daily  counts  less  than  three 
for  both  outpatient  visits  and  laboratory  tests  in  all  regions.  A  time  series  plot  of 
outpatient  visits  with  a  neurological  syndrome  diagnosis  in  the  national  capital  area 
shows  what  may  be  an  outbreak  beginning  in  August  2003,  with  an  initial  alert  on  August 
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18  and  six  more  alerts  up  to  and  including  the  peak  count  of  15  on  September  16  (figure 
5.6).  During  this  entire  period,  however,  the  daily  number  of  laboratory  tests  for 
neurological  syndrome  never  exceeded  two. 

At  Camp  LeJeune,  the  outpatient  visit  data  for  neurological  syndrome  show  an 
increase  in  September  2004  with  a  single-day  alert  on  September  21  (figure  5.6). 

Laboratory  test  orders  for  neurological  syndrome  alerted  on  September  2  and  3,  nearly 
three  weeks  before  the  alert  in  the  outpatient  visit  data.  This  may  be  an  early  indication 
for  the  second  alert  period  noted  in  the  outpatient  visit  data,  or  it  may  be  unrelated. 

While  laboratory  test  orders  provide  some  corroboration  for  the  observed  increase  in  the 
outpatient  visit  data,  they  also  have  a  higher  background  alerting  rate  than  outpatient 
visits  even  though  the  significance  level  for  alerting  is  set  the  same  for  both  data  streams. 

One  possible  explanation  is  neurological  tests  are  often  ordered  in  multiples.  Of  the 
10,940  individuals  in  this  data  set  for  whom  neurological  tests  were  ordered,  5,430  (50%) 
had  more  than  one  neurological  test  ordered  at  the  same  MTF  on  the  same  day.  Frequent 
combinations  were  CSF  cultures  and  gram  stains  of  the  cerebrospinal  fluid  (n=3,387)  and 
two  separate  CSF  cultures  (n  =  1,660).  This  problem  could  be  addressed  by  screening 
out  multiple  tests  before  applying  the  surveillance  algorithm.  Screening  should  be 
applied  to  all  syndromes,  but  is  unlikely  to  have  much  effect  on  syndromes  other  than 
neurologic  because  for  other  syndromes  usually  only  one  test  in  each  syndrome  is 
ordered  per  person  at  the  same  MTF  on  the  same  day. 

Fever  syndrome. 

Fever  data  follow  a  seasonal  pattern  similar  to  that  observed  in  respiratory  data, 
perhaps  because  many  winter  fevers  may  be  influenza-related.  For  this  case  study  we 
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consider  the  winter  2003-2004  season  in  three  regions:  Camp  LeJeune,  San  Diego,  and 
the  national  capital  area  (Figure  5.7).  Outpatient  visits  for  fever  in  Camp  LeJeune  show  a 
distinct  increase  beginning  in  late  October,  with  an  initial  alert  on  October  27  and  seven 
more  alerts  before  the  peak  in  late  November.  Laboratory  test  orders  also  show  an 
increase  in  late  October  with  a  flatter  peak.  The  first  alert  in  the  laboratory  test  order  data 
is  two  days  after  the  first  alert  in  the  outpatient  visit  data,  on  October  29,  followed  by 
four  more  alerts  in  the  next  eight  days. 

In  the  national  capital  area,  while  both  data  sources  show  a  winter  increase  in 
fever  syndrome  counts,  outpatient  visits  alert  on  November  17,  nearly  one  month  before 
the  first  alert  in  laboratory  test  order  data  on  December  10.  In  San  Diego,  outpatient 
visits  show  an  initial  alert  on  November  12,  followed  by  a  sustained  alerting  period  from 
November  28  to  December  23.  Laboratory  data  show  only  one  alert,  a  single-day  spike 
on  December  18  near  the  peak  of  the  increase  in  outpatient  visits. 

These  results  suggest  that  laboratory  test  order  data  may  be  less  useful  than 
outpatient  visit  data  for  early  detection  of  fever  outbreaks,  but  may  be  useful  for 
corroborating  seasonal  increases  in  fever  syndrome. 

Summary 

Despite  the  short  time  frame  and  lack  of  documented  outbreaks  for  evaluating 
syndromic  surveillance  using  laboratory  test  order  data,  these  results  suggest  that 
laboratory  test  orders  may  be  a  useful  adjunct  to  outpatient  visit  data  for  detecting  and 
monitoring  disease  outbreaks.  Among  the  findings; 
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•  Although  more  respiratory  and  gastrointestinal  outbreaks  were  detected  using 
outpatient  visit  data,  laboratory  test  orders  detected  an  outbreak  that  was  not 
identified  in  the  outpatient  visit  data. 

•  Surveillance  using  a  simple  EWMA  algorithm  performed  slightly  better  than 
the  DoD-ESSENCE  algorithm  with  the  laboratory  test  order  data. 

•  Combining  the  two  data  sources  may  yield  earlier  alerts. 

•  Case  studies  indicate  that  laboratory  test  orders  may  be  useful  for  monitoring 
rash,  neurological  and  fever  syndromes. 

A  further  advantage  of  laboratory  test  order  data  is  the  collection  mechanism.  A 
computer  record  of  the  laboratory  test  order  is  typically  produced  at  the  time  of  order, 
usually  during  the  outpatient  visit.  Although  these  initial  records  are  not  currently 
archived,  it  would  be  theoretically  possible  to  develop  a  system  to  retrieve  these  records 
in  real  time.  Thus  alerts  might  occur  earlier  in  the  laboratory  data  than  in  the  outpatient 
visit  data  which  are  often  recorded  three  or  more  days  after  the  encounter.  Ultimately, 
this  advantage  will  disappear  as  the  DoD  transitions  to  newer  versions  of  CHCS  that  will 
combine  clinical  and  laboratory  information  in  a  single  record.  Despite  the  limitations  of 
this  study,  there  is  sufficient  evidence  to  recommend  prospective  surveillance  using 
laboratory  test  data,  with  further  research  and  evaluation  to  fine-tune  the  approach. 
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Figure  5.1:  Respiratory  outbreaks 
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Figure  5.2:  Gastrointestinal  outbreaks 


Figure  5.5.  Rash  case  studies 
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Figure  5.6  Neurological  case  studies 
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Figure  5. 7.  Fever  case  studies 


CHAPTER  VI:  Discussion  and  conclusion 


This  study  has  shown  that  data  on  laboratory  test  orders,  currently  collected  and 
archived  for  administrative  purposes,  may  be  useful  as  a  supplementary  data  source  for 
syndromic  surveillance.  Chapter  6  will  summarize  and  discuss  the  major  findings  of  the 
study,  address  specific  findings  that  may  facilitate  incorporation  of  laboratory  test  order 
data  into  ESSENCE,  discuss  limitations  of  the  study  and  how  to  address  them  in  future 
research,  propose  next  steps  for  research,  and  place  this  work  within  a  broader  context  of 
disease  surveillance  and  public  health  research. 

Major  results 

The  goal  of  the  first  phase  of  the  study  was  to  develop  and  validate  syndrome 
definitions  comparable  to  the  ICD-9  based  syndrome  definitions  currently  used  by 
ESSENCE.  The  first  task  was  to  develop  a  consistent  nomenclature  useful  for  syndromic 
surveillance.  Different  laboratories  use  different  nomenclature  (e.g.  “THR  CULT”  and 
“THROAT  C”  for  throat  culture)  and  some  laboratory  test  names  are  quite  general  (e.g. 
“miscellaneous  culture”,  “other  culture/sensitivity”).  This  lack  of  consistency  has  been  a 
principal  barrier  to  automated  analysis  of  laboratory  test  records.  It  took  several  months 
for  our  panel  of  three  researchers  to  develop  a  list  of  standard  test  names  and  map  the  test 
names  in  the  database  to  the  standard  list.  Although  there  is  undoubtedly  some 
misclassification  in  this  scheme,  we  reviewed  the  specimen  sources  associated  with  each 
test  name  and  found  that  they  were  largely  consistent.  For  example,  99.5  percent  of 
1 1,000  cerebrospinal  fluid  cultures  listed  a  specimen  source  of  cerebrospinal  fluid  and 
99.6  percent  of  650,000  throat  cultures  listed  a  specimen  source  of  throat  or  pharynx  (the 
rest  were  body  fluid,  swab,  or  other). 
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The  standard  test  names  are  too  granular  for  syndromic  surveillance  for  two 
reasons.  First,  syndromic  surveillance  of  71  test  names  would  be  unwieldy  and  yield  too 
many  false  alerts.  Second,  even  after  the  mapping  to  standard  nomenclature,  some  tests 
may  appear  under  more  than  one  name.  Some  labs,  for  example,  listed  specific  tests  for 
salmonella,  shigella  and  Campylobacter  under  these  names  while  others  listed  them  under 
the  generic  “stool  culture”  category.  Surveillance  using  the  test  name  “Salmonella” 
would  miss  a  large  proportion  of  tests  in  which  Salmonella  is  a  suspected  pathogen.  For 
useful  surveillance,  the  test  names  should  be  combined  into  syndrome  groups. 

The  second  task,  therefore,  was  to  construct  and  validate  syndrome  groups.  We 
had  three  sources  of  information  against  which  to  validate  the  syndrome  groups: 
syndromic  data  based  on  outpatient  ICD-9  diagnostic  codes  for  the  same  patients,  timing 
and  location  of  known  outbreaks,  and  syndrome  groups  developed  by  a  CDC  panel  of 
physicians  and  epidemiologists  [35].  Although  none  of  the  three  sources  can  be 
considered  a  true  “gold  standard”,  we  attempted  to  construct  syndromes  that  are  roughly 
consistent  with  all.  We  assigned  a  laboratory  test  to  a  specific  syndrome  based  on  three 
criteria: 

•  The  test  was  preferentially  ordered  during  outpatient  visits  for  the  syndrome. 

•  Daily  counts  of  test  orders  were  positively  correlated  over  time  with  daily  counts 
of  outpatient  visits  for  the  syndrome  in  several  regions. 

•  For  respiratory  and  GI  syndromes,  the  number  of  orders  of  the  laboratory  test 
increased  during  known  outbreaks  of  the  syndrome. 

The  resulting  syndromes  were  consistent  with  the  syndromes  developed  by  the 
CDC  panel,  with  a  few  exceptions  discussed  in  Chapter  4. 
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Once  the  syndromes  were  developed  and  validated,  the  second  goal  of  the  study 
was  to  evaluate  the  usefulness  of  laboratory  test  orders  for  syndromic  surveillance.  The 
primary  outcome  measures  were  the  number  of  outbreaks  (out  of  six)  that  were  detected 
before  they  would  have  been  recognized  by  traditional  public  health  surveillance,  and  the 
number  of  days  between  the  start  of  the  outbreak  and  detection  by  syndromic 
surveillance.  Three  main  comparisons  were  reported:  surveillance  using  laboratory  test 
orders  vs.  surveillance  using  outpatient  visit  diagnoses,  using  the  DoD-ESSENCE 
algorithm  for  alerting  in  both  cases;  surveillance  using  the  DoD-ESSENCE  algorithm  vs. 
surveillance  using  other  commonly-used  surveillance  algorithms,  using  the  laboratory  test 
order  data  in  all  cases,  and  surveillance  using  laboratory  data  combined  with  outpatient 
visit  diagnoses  vs.  outpatient  visit  diagnoses  alone,  using  the  DoD-ESSENCE  algorithm 
and  simple  combination  of  p-values  for  combined  alerting. 

Using  an  alert  rate  of  one  per  six  weeks,  outpatient  visit  data  detected  more 
outbreaks  (four  of  six)  than  laboratory  test  order  data  (two  of  six).  Time  to  alert  was 
similar  for  the  two  data  sources:  15  days  for  outpatient  visits  data  and  17  days  for 
laboratory  test  order  data.  These  limited  findings  suggest  that  laboratory  test  orders  may 
be  useful  for  syndromic  surveillance,  but  are  unlikely  to  replace  outpatient  visits  as  a  data 
source. 

One  reason  laboratory  test  orders  may  be  less  sensitive  than  outpatient  visits  for 
outbreak  detection  in  this  study  is  that  during  some  outbreaks,  laboratory  tests  would  be 
ordered  only  if  identifying  the  pathogen  served  an  important  clinical  or  public  health 
function.  During  a  gastrointestinal  outbreak,  for  example,  clinicians  may  be  unlikely  to 
order  a  stool  culture  unless  the  symptoms  are  unusual.  If  this  is  the  case,  sensitivity  of 
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laboratory  test  orders  for  outbreak  detection  might  be  increased  during  an  outbreak  of  a 
novel  pathogen  with  unusual  symptoms.  On  the  other  hand,  if  laboratory  tests  are 
ordered  more  often  for  unusual  illnesses,  this  data  source  may  be  more  specific  than 
outpatient  visits  for  detecting  outbreaks  outside  of  seasonal  respiratory  and 
gastrointestinal  illness.  It  is  not  possible  to  measure  specificity  directly  in  this  study 
because  for  most  alerts  we  cannot  determine  retrospectively  whether  an  outbreak  was 
present  or  whether  the  alert  corresponds  to  a  “false  positive.” 

The  DoD-ESSENCE  algorithm  performed  fairly  well  with  the  laboratory  test 
order  data,  although  there  is  some  evidence  that  a  simple  EWMA  algorithm  would 
perform  even  better.  One  topic  for  future  research  would  be  to  evaluate  whether 
adjusting  the  parameters  of  the  DoD-ESSENCE  algorithm  would  improve  its 
performance  relative  to  EWMA  with  laboratory  data. 

When  /^-values  from  the  two  data  sources  were  combined,  three  of  six  outbreaks 
were  detected,  compared  with  two  of  six  from  laboratory  test  orders  alone  and  four  of  six 
from  outpatient  visit  data  alone.  However,  median  lag  time  from  the  start  of  the  outbreak 
to  the  first  alert  improved  markedly,  from  roughly  two  weeks  for  each  data  source 
separately  to  four  days  for  the  combined  data.  Although  it  would  be  premature  to  draw  a 
firm  conclusion  based  on  six  outbreaks,  our  results  indicate  that  combining  data  from 
multiple  data  sources  may  yield  a  substantial  improvement  in  timeliness  of  outbreak 
detection. 

Specific  findings 

In  addition  to  addressing  the  general  questions  discussed  above,  this  study  was 
able  to  shed  light  on  several  specific  issues: 
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1.  Should  surveillance  of  laboratory  tests  focus  on  specific  tests,  syndrome  groups  or 
both? 

One  way  to  conduct  surveillance  of  laboratory  test  orders  would  be  to  monitor  the 
frequency  of  specific  tests  of  interest.  This  is  especially  appealing  with  respect  to  tests 
for  smallpox,  anthrax,  and  other  potential  agents  of  bioterrorism  which  might  get  lost  in 
the  noise  of  larger  syndromes.  The  laboratory  test  orders  for  these  conditions  are  likely 
to  be  much  more  specific  than  ICD-9  diagnosis  codes  based  on  symptoms  and 
unconfirmed  diagnoses.  However,  there  are  problems  with  this  approach  under  the 
current  system.  The  inconsistent  terminology  and  infrequency  of  these  tests  means  that 
we  do  not  know  how  an  anthrax  test  will  be  listed  when  it  appears  in  the  data  file.  The 
test  order  name  could  somehow  mention  anthrax,  but  multiple  abbreviations  are  possible. 

More  likely,  rare  tests  would  be  listed  as  “miscellaneous  culture”  or  “other”,  and  anthrax 
would  be  mentioned  only  in  the  results.  As  an  adjunct  to  daily  counts  of  tests  in  each 
syndrome  group,  it  might  be  useful  to  scan  all  records  for  certain  keywords  such  as 
“anthrax”,  using  an  algorithm  that  can  account  for  different  spellings  and  abbreviations. 

This  approach  should  not  replace  syndromic  surveillance,  however.  Smallpox,  for 
example,  might  show  up  in  respiratory  or  fever  syndromes  under  several  different  test 
names  because  of  the  nonspecific  prodrome. 

2.  Should  the  syndromes  be  based  on  the  test  name  or  specimen  source? 

An  early  proposal  for  dealing  with  the  lack  of  consistent  nomenclature  was  to 
ignore  the  test  names  and  group  tests  into  syndrome  by  specimen  source.  Under  this 
proposal  all  blood  specimens  would  correspond  to  the  fever  syndrome,  all  stool 
specimens  to  the  GI  syndrome,  all  throat  cultures  would  correspond  to  respiratory,  and  all 
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skin  cultures  to  rash.  While  this  might  yield  a  reasonable  approximation  of  the 
syndrome  groups,  the  data  suggest  that  the  specimen  source  field  would  be  less  accurate 
than  the  test  order  field.  Gastrointestinal  and  respiratory  tests  based  on  specimen  source 
would  include  tests  that  are  not  of  interest  for  syndromic  surveillance.  For  example,  35% 
of  tests  with  specimen  source  of  “rectum”  correspond  to  GC  cultures,  group  B  strep,  or 
herpes  tests  rather  than  stool  cultures.  By  comparison,  at  least  95  percent  of  tests  with  the 
name  “stool  culture”  were  obtained  from  rectal/fecal  specimens.  There  are  over  300  tests 
with  a  specimen  source  of  “throat”  but  a  test  name  of  GC  culture;  all  tests  with  the  name 
“throat  culture”,  on  the  other  hand,  had  specimen  source  listed  as  throat  specimen  or  as 
body  fluid/other/swab.  The  rash  syndrome  would  fail  to  include  ten  percent  of  skin 
cultures  if  the  definition  were  based  on  specimen  source.  About  90  percent  of  skin 
cultures  include  the  word  “skin”  in  the  specimen  source  field,  while  the  rest  describe  the 
specific  body  site,  e.g.  “scalp”,  “left  leg”. 

3.  Should  daily  counts  be  monitored  at  the  MTF  or  regional  level? 

At  the  clinic  level,  most  syndromes  have  very  few  laboratory  tests.  Appendix  2 
shows  average  daily  counts  of  all  tests  for  each  MTF  included  in  the  study.  Counts  by 
syndrome  are  even  lower,  with  average  frequency  of  test  orders  less  than  one  per  day  in 
nine  out  of  ten  clinics  for  fever,  GI,  neurological,  and  rash  syndromes.  Almost  half  of 
clinics  have  average  daily  counts  less  than  one  for  respiratory  syndrome,  the  most 
prevalent  syndrome.  Flospitals,  on  the  other  hand,  make  up  less  than  one-quarter  of 
MTFs  in  this  study  but  account  for  nearly  60  percent  of  test  orders.  It  seems  likely  that 
monitoring  laboratory  test  orders  at  every  MTF  is  likely  to  result  in  many  false  alarms 
based  on  tiny  increases  in  laboratory  test  orders.  At  a  facility  in  which  a  particular 
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laboratory  test  is  rarely  ordered,  even  a  single  order  could  trigger  an  alert.  A  more 
efficient  solution  may  be  to  monitor  outpatient  tests  at  hospitals.  These  may  represent 
patients  with  more  severe  illness,  and  the  volume  of  tests  is  large  enough  that  patterns  can 
emerge.  Even  better,  combining  data  from  a  hospital  and  its  affiliated  clinics  can  give  a 
regional  picture.  One  concern  is  that  localized  outbreaks  may  manifest  first  at  local 
clinics  and  may  be  missed  by  regional  surveillance.  This  may  be  an  acceptable  tradeoff 
for  avoiding  numerous  false  alerts,  although  during  periods  of  heightened  concern  it  may 
be  useful  to  monitor  at  the  clinic  level. 

4.  How  should  tests  be  counted? 

The  structure  of  the  HL7  files  can  lead  to  overcounting  of  laboratory  test  orders  in 
two  ways.  First  many  duplicate  messages  are  included  in  the  file.  This  is  a  technical 
issue  that  is  easily  addressed  by  searching  for  and  deleting  records  with  duplicate 
message  IDs.  However,  failure  to  follow  this  step  can  yield  misleading  results.  A  more 
difficult  issue  is  that  both  “pending”  and  “final”  results  are  sent  for  many  tests.  Pending 
results  account  for  one-quarter  of  the  records  in  our  data  set,  and  each  of  these  has  a 
corresponding  “final”  record  with  a  different  identifier.  Since  this  study  relied  on 
archived  data  we  considered  only  “final”  results.  “Pending”  results  would  be  a  better 
choice  for  ongoing  surveillance  when  available  because  they  are  available  sooner. 
However,  care  must  be  taken  to  screen  out  the  “final”  results  corresponding  to  tests  that 
have  already  been  counted  at  the  “pending”  stage.  A  better  approach  would  be  to  treat 
the  test  order  as  its  own  record,  separate  from  the  results.  This  would  eliminate  the  need 
to  consider  pending  and  final  records,  and  could  also  make  the  test  order  information 


101 


available  at  the  time  of  order  without  the  need  to  wait  for  certification  of  results  in  the 
laboratory. 

Multiple  tests  per  person  per  day  for  the  same  syndrome  also  can  lead  to 
overcounting  and  false  alerts.  As  discussed  in  chapter  5,  half  of  the  patients  for  whom  a 
neurological  test  was  ordered  had  more  than  one  neurological  test  at  the  same  MTF  at  the 
same  day.  Since  neurological  tests  are  rare,  two  tests  for  the  same  person  in  a  single  day 
may  trigger  an  alert.  Multiple  tests  are  less  of  a  problem  in  the  other  syndromes. 

Multiple  tests  are  ordered  for  29  percent  of  fever  patients,  35  percent  of  GI  patients,  15 
percent  of  rash  patients,  and  6  percent  of  respiratory  patients.  While  we  did  not  leave  out 
multiple  tests  in  this  study,  it  would  be  best  to  do  so  in  future  research. 

Limitations 

The  study  has  several  limitations,  mostly  related  to  inadequacies  in  the  data. 

1.  No  gold  standard  for  validation  of  syndromes. 

Two  of  the  three  validation  criteria  in  Chapter  4  were  based  on  outpatient  visit 
syndromes.  Laboratory  test  orders  are  considered  to  be  associated  with  a  syndrome  if 
they  are  preferentially  ordered  during  outpatient  visits  for  that  syndrome,  and  if  daily 
counts  of  laboratory  tests  orders  are  correlated  over  time  with  daily  counts  of  outpatient 
visits  for  a  particular  syndrome.  However,  there  is  no  guarantee  that  syndromes  based  on 
ICD-9  diagnosis  codes  reflect  actual  illness  with  a  particular  syndrome.  Use  of  ICD-9 
codes  can  be  highly  idiosyncratic.  The  gold  standard  for  assigning  a  patient  to  a 
particular  syndrome  is  physician  chart  review.  A  previous  study  by  Foster  et  al.  [13] 
showed  that  across  three  syndromes  and  several  emergency  departments,  classification  of 
outpatient  visits  based  on  ICD-9  diagnosis  codes  agreed  closely  with  physician  chart 
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review.  Most  patients  diagnosed  with  a  syndrome  based  on  chart  review  also  were 
assigned  to  that  syndrome  based  on  ICD-9  codes  (sensitivity,  67-95  percent),  and  few 
patients  were  assigned  to  a  syndrome  based  on  their  ICD-9  diagnosis  codes  without 
corresponding  symptoms  noted  in  their  chart  (specificity,  92-97  percent).  So  despite  the 
impossibility  of  using  chart  reviews  as  a  gold  standard  for  this  study,  using  ICD-9-based 
syndromes  as  a  surrogate  seems  reasonable. 

2.  Too  few  outbreaks. 

In  this  study,  four  confirmed  outbreaks  were  used  to  validate  the  syndrome 
groupings,  and  six  outbreaks  were  used  to  evaluate  sensitivity  and  timeliness  of  outbreak 
detection.  All  of  these  outbreaks  corresponded  to  respiratory  or  gastrointestinal 
syndromes,  so  we  could  not  evaluate  the  remaining  syndromes  with  respect  to  confirmed 
outbreaks.  Many  more  outbreaks  would  be  needed  to  obtain  reliable  estimates  of 
sensitivity  and  timeliness  even  for  respiratory  and  gastrointestinal  syndromes. 

It  was  not  possible  to  identify  more  outbreaks  because  the  data  available  for  the 
study  correspond  to  a  two-year  period  that  ended  two  years  before  the  analysis.  With 
current  data,  it  might  have  been  possible  to  identify  increases  in  outpatient  visits  or 
laboratory  test  orders,  and  then  contact  MTFs  to  determine  whether  the  increases 
represented  actual  outbreaks  or  not.  But  contacting  MTFs  about  potential  outbreaks  that 
occurred  more  than  two  years  ago  did  not  seem  like  a  fruitful  strategy. 

One  way  to  overcome  this  limitation  would  be  to  use  simulated  outbreaks.  Using 
this  approach,  extra  laboratory  test  orders  would  be  injected  into  either  actual  data  or 
simulated  data  representing  the  background  rate  of  laboratory  test  orders  in  the  absence 
of  an  outbreak.  The  distribution  of  the  injected  data  would  be  based  on  models  of  disease 
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exposure,  incubation  periods,  and  expected  health  care  utilization  (e.g.  Coberley  et  al. 
[71]).  The  missing  piece  of  the  simulation  model  for  this  study  is  health  care  utilization. 
Modeling  the  proportion  of  ill  persons  who  would  have  an  outpatient  visit,  and  the 
proportion  of  those  with  an  outpatient  visit  who  would  get  a  particular  laboratory  test,  is 
beyond  the  scope  of  this  study. 

As  part  of  the  data  analysis  presented  in  Chapter  4,  we  calculated  the  proportion 
of  outpatient  visits  in  each  ICD-9  based  syndrome  that  have  a  corresponding  laboratory 
test  for  that  syndrome  (data  not  shown).  This  could  be  an  important  part  of  building  a 
model  for  simulating  disease  outbreaks  in  laboratory  test  order  data.  Physicians  typically 
order  one  fever  test  per  102  fever  visits,  one  GI  test  per  63  GI  visits,  one  neurological  test 
per  27  neurological  visits,  one  rash  test  per  450  rash  visits,  and  one  respiratory  test  per  18 
respiratory  visits.  However,  we  still  do  not  know  whether  the  proportions  would  change 
during  an  outbreak.  It  seems  likely  that,  once  an  outbreak  diagnosis  is  established,  test 
ordering  will  decrease  even  if  the  outbreak  is  ongoing,  because  new  cases  wil  be  assumed 
to  be  part  of  the  outbreak.  The  Charleston  GI  outbreak  described  in  chapter  5  showed 
this  pattern,  with  tests  “bottoming  out”  for  several  days  after  the  public  health  recognition 
date.  Future  research  could  investigate  whether  this  is  a  consistent  pattern. 

3.  Misclassification  of  tests. 

Tests  may  be  misclassified  on  two  levels.  First,  the  test  name  on  the  laboratory 
test  record  may  be  assigned  to  the  wrong  standardized  test  name.  For  example,  while  the 
vast  majority  of  laboratory  tests  assigned  to  the  “CSF  CULTURE”  standardized  test 
name  listed  cerebrospinal  fluid  as  the  specimen  source,  there  were  ten  “CSF  CULTURE” 
tests  that  listed  the  specimen  source  as  “blood”.  It  is  impossible  to  determine  whether  the 
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test  name  or  the  specimen  source  is  in  error,  so  for  some  of  these  cases,  they  may  be 
incorrectly  classified  as  CSF  cultures.  Although  this  type  of  mismatch  is  relatively  rare, 
it  illustrates  that  the  data  set  does  not  always  provide  sufficient  or  consistent  information 
to  assign  tests  to  the  appropriate  category. 

Second,  tests  may  be  assigned  to  the  wrong  syndrome  group.  For  example,  over 
3,000  tests  are  listed  as  “miscellaneous  culture”  or  similar  on  the  test  records.  These  tests 
are  only  assigned  to  a  syndrome  if  the  specimen  source  falls  unambiguously  into  one  of 
the  syndromes  (blood  for  fever,  stool  for  GI,  CSF  for  neurological,  throat/pharynx  for 
respiratory.)  The  70  cultures  with  specimen  source  of  “leg”  or  “scalp”  are  probably  skin 
or  wound  cultures  that  could  be  assigned  to  the  rash  syndrome  if  more  information  were 
available. 

A  potential  consequence  of  misclassification  is  that  it  could  attenuate  the  signals 
seen  in  the  data  relative  to  the  noise.  Small  to  moderate  increases  in  laboratory  tests  for  a 
particular  syndrome  will  appear  even  smaller  if  many  of  the  laboratory  tests  associated 
with  the  increase  are  not  included  in  the  appropriate  syndrome.  Standardizing  laboratory 
test  order  nomenclature,  for  both  test  names  and  specimen  sources,  can  go  a  long  way 
towards  fixing  this  problem.  In  the  mean  time,  it  appears  that  even  with  the  potential 
misclassification,  laboratory  test  order  data  are  still  useful  for  detecting  outbreaks. 

4.  Data  include  only  microbiology  laboratory  tests. 

Test  orders  are  broadly  categorized  as  microbiology,  chemistry,  or  radiology  in 
the  HL-7  records.  This  study  evaluated  only  microbiology  data  because  the  total  volume 
of  test  records  was  so  large  that  processing  the  archived  data  would  have  been 
prohibitive,  and  microbiology  tests  are  expected  to  be  most  specific  for  infectious 
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diseases.  However,  particular  chemistry  tests  may  be  useful  for  monitoring  infectious 
disease  syndromes.  One  of  the  most  promising  candidates  is  rapid  influenza  tests.  Asha 
Riegododios  at  the  Navy  Environmental  Health  Center  has  plotted  daily  counts  of  these 
tests  for  one  MTF  and  they  show  a  stronger  seasonal  pattern  than  is  observed  in  throat 
cultures  (personal  correspondence).  Obtaining  data  on  additional  tests  should  be  a  high 
priority  for  future  research. 

5.  Incomplete  data. 

The  time  frame  for  this  study  occurred  before  many  of  the  MTFs  began 
submitting  laboratory  test  records  to  the  HF7  archive.  Air  Force  facilities  in  particular 
were  less  likely  to  participate  before  the  fall  of  2004.  Thus  the  sample  is  heavily 
weighted  towards  Army  and  Navy  MTFs.  The  assignment  of  laboratory  test  names  to 
standardized  categories  will  have  to  be  re-evaluated  in  current  data  to  ensure  that 
laboratory  test  names  used  in  the  MTFs  that  came  online  after  the  study  ended  are 
assigned  to  the  appropriate  categories.  This  problem  can  be  resolved  over  time  as  MTFs 
become  more  familiar  with  the  system  and  adopt  a  standardized  nomenclature. 

Future  research 

Several  topics  for  future  research  were  mentioned  above:  continuing  the  study 
with  more  outbreaks  and  current  data  to  determine  whether  the  findings  hold  up, 
developing  a  model  of  test  ordering  behavior  and  using  the  model  to  simulate  outbreaks, 
adjusting  the  parameters  of  the  DoD-ESSENCE  algorithm,  and  incorporating  data  on 
chemistry  tests.  Several  additional  research  questions  may  also  be  of  interest 
1.  Is  immediate  electronic  capture  of  test  orders  feasible? 
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This  study  suggests  that  if  laboratory  data  are  to  improve  timeliness  of  outbreak 
detection,  much  of  that  improvement  must  come  from  earlier  electronic  capture  of 
laboratory  test  order  records.  Laboratory  tests  are  generally  ordered  during  an  outpatient 
visit,  so  they  are  unlikely  to  show  an  earlier  signal  than  outpatient  visits  otherwise.  The 
fact  that  laboratory  tests  are  typically  ordered  electronically  indicates  that  it  should  be 
feasible  to  capture  the  electronic  record  at  the  time  of  order.  It  would  be  worthwhile  to 
determine  the  modifications  to  the  computer  system  that  would  be  required  to  implement 
immediate  capture  of  laboratory  test  orders. 

2.  Are  daily  counts  the  best  measure  of  laboratory  test  orders? 

The  results  of  this  study  suggest  that  joint  surveillance  of  both  outpatient  visits 
and  laboratory  test  orders  can  improve  the  timeliness  of  outbreak  detection.  The  method 
used  was  a  simple  combination  of  p-values,  and  a  more  sophisticated  method  for 
combining  data  might  yield  better  results.  One  assumption  of  many  methods  for 
multivariate  surveillance  is  that  the  data  streams  are  independent.  Since  nearly  all 
laboratory  tests  are  ordered  during  an  outpatient  visit,  this  assumption  must  be  false.  One 
suggestion  for  addressing  this  issue  is  to  monitor  the  ratio  of  laboratory  tests  to  outpatient 
visits  rather  than  daily  counts  of  outpatient  visits.  This  suggestion  is  intuitively  appealing 
because  it  may  be  that  physicians  order  tests  for  a  higher  proportion  of  their  patients 
during  an  outbreak  of  an  unknown  illness.  A  future  research  project  could  evaluate 
which  approach  leads  to  better  outbreak  detection. 

3.  Does  linking  laboratory  tests  with  corresponding  outpatient  visits  yield  better 
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As  described  above,  it  may  be  useful  to  monitor  the  ratio  of  laboratory  tests  to 
outpatient  visits.  This  could  be  done  by  calculating  daily  counts  separately  from  each 
data  source  and  taking  a  simple  ratio.  An  extension  of  this  research  question  is  whether  it 
would  be  even  better  to  monitor  the  proportion  of  tests  for  a  particular  syndrome  in  which 
a  laboratory  test  for  that  syndrome  is  ordered.  Similarly,  it  might  be  useful  to  monitor 
daily  counts  of  patients  with  both  an  outpatient  visit  and  a  laboratory  test  order  for  the 
same  syndrome.  This  would  require  linking  laboratory  tests  to  the  outpatient  visits 
during  which  they  were  ordered.  Then,  for  example,  throat  cultures  that  were  ordered 
without  a  corresponding  respiratory  diagnosis  would  not  be  counted.  The  analysis  in 
Chapter  4  demonstrates  that  this  linking  is  feasible,  if  computationally  intensive.  It 
would  be  worthwhile  to  evaluate  whether  linking  records  would  yield  a  gain  in  outbreak 
detection  that  outweighs  the  additional  complexity. 

4.  Do  specific  population  subgroups  provide  earlier  indication  of  outbreaks? 

Demographic  subgroups,  specifically  children,  have  been  shown  to  be  sentinel 
populations  for  influenza  [72],  An  unpublished  analysis  of  outpatient  visit  records  used 
in  DoD-ESSENCE  indicated  that  when  outpatient  visits  for  respiratory  syndrome  are 
analyzed  separately  by  age  group,  seasonal  increases  do  not  occur  earlier  in  children  than 
in  adults.  Because  of  the  inconsistent  findings  across  data  sets,  it  would  be  worthwhile  to 
evaluate  whether  outbreaks  alert  earlier  in  the  laboratory  test  data  for  specific  age  groups. 

5.  Can  automated  surveillance  be  performed  on  laboratory  test  results? 

Surveillance  of  laboratory  test  results  would  not  be  particularly  useful  for  early 
detection  of  outbreaks  because  of  the  time  delay  between  onset  of  symptoms  and 
certification  of  results.  However,  automated  surveillance  of  laboratory  test  results  would 
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be  tremendously  useful  in  other  ways.  During  an  outbreak,  automated  surveillance  could 
identify  cases  who  could  then  be  contacted  as  part  of  an  epidemiological  investigation. 
Identifying  the  pathogen  responsible  for  the  outbreak  would  be  facilitated  by  having  an 
electronic  record  of  all  test  results  during  the  time  period.  Reportable  diseases  could  be 
monitored  automatically  using  existing  records,  potentially  eliminating  the  need  for  a 
separate  reporting  system  and  reducing  the  reporting  burden  on  physicians  and 
laboratories.  Research  of  this  type  is  ongoing  at  the  Navy  Environmental  Health  Center, 
and  much  more  is  needed. 

Public  health  significance 

This  study  makes  several  novel  contributions  to  public  health.  This  is  the  first 
systematic  evaluation  of  DoD  laboratory  test  data  for  general  public  health  surveillance. 
While  Riegododios  et  al  have  used  DoD  laboratory  data  for  surveillance  of  specific 
illnesses  [31-34],  this  study  complements  that  research  by  expanding  the  focus  to 
syndromic  surveillance.  The  CDC  has  incorporated  laboratory  test  data  from  LabCorp,  a 
nationwide  laboratory  system  for  syndromic  surveillance  that  tests  more  than  340,000 
specimens  daily  (Ma  et  al.,  2005).  This  is  the  only  other  study  of  a  nationwide  laboratory 
data  source  for  syndromic  surveillance,  the  first  attempt  to  validate  such  a  source  against 
outpatient  visit  data,  and  the  first  to  evaluate  outbreak  detection  using  laboratory  data. 

One  stated  goal  of  syndromic  surveillance  is  “to  enable  earlier  detection  of 
epidemics  and  a  more  timely  public  health  response,  hours  or  days  before  disease  clusters 
are  recognized  clinically,  or  before  specific  diagnoses  are  made  and  reported  to  public 
health  authorities”  [1],  Early  syndromic  surveillance  systems  were  developed 
specifically  to  detect  bioterrorist  attacks  such  as  anthrax  and  smallpox.  Since  then,  public 
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health  attention  has  become  more  focused  on  emerging  infectious  diseases,  including 
SARS  and  pandemic  influenza.  Regardless  of  the  source  of  the  threat,  early  detection  of 
outbreaks  can  lead  to  early  intervention  and  mitigate  the  costs  of  the  outbreak.  This  study 
is  the  first  to  show  that  incorporating  laboratory  test  orders  into  an  existing  surveillance 
system  can  improve  the  timeliness  of  outbreak  detection. 

While  early  detection  is  still  a  primary  goal,  research  in  syndromic  surveillance 
has  expanded  to  include  ongoing  situational  awareness.  The  CDC,  for  example,  now 
refers  to  its  BioSense  program  as  a  system  for  “early  event  detection  and  situational 
awareness.”  Once  an  outbreak  is  detected,  syndromic  surveillance  can  monitor  whether 
the  number  of  cases  is  increasing  or  decreasing,  describe  the  demographic  and 
geographic  distribution  of  cases,  and  provide  other  useful  information  without  adding  to 
the  workload  of  clinicians.  This  evaluation  of  laboratory  test  data  is  an  important  first 
step  towards  being  able  to  monitor  not  just  syndromes,  but  laboratory-confirmed 
diagnoses,  during  an  outbreak. 

Electronic  records  have  the  potential  to  revolutionize  public  health  research. 
Because  the  data  are  collected  anyway  for  clinical  and  billing  purposes,  there  is  no 
additional  burden  for  clinical  staff  to  collect  data  specifically  for  research.  Rare  illness 
may  be  studied  because  the  huge  quantity  of  administrative  records  makes  it  possible  to 
identify  many  people  with  a  specific  illness,  even  if  a  specific  provider  may  only  see  a 
handful  of  such  patients.  Linkage  of  hospital,  clinic,  pharmacy  and  laboratory  records 
makes  it  possible  to  gain  a  thorough  clinical  picture  and  track  patients  through  the  system 
over  time.  Laboratory  data  may  useful  not  only  for  syndromic  surveillance,  but  as  part  of 
an  overall  trend  towards  leveraging  administrative  data  for  public  health  research.  While 
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there  is  much  work  left  to  be  done,  this  study  shows  that  it  is  feasible  to  obtain  useful 
public  health  information  from  DoD  laboratory  test  data,  and  to  link  this  information  to 
other  claims  for  the  same  patients. 

In  summary,  this  study  demonstrates  that  laboratory  test  order  data  can  improve 
the  performance  of  the  DoD-ESSENCE  syndromic  surveillance  system.  Laboratory  test 
data  will  undoubtedly  prove  useful  for  many  other  types  of  public  health  research. 

Surveillance  and  research  would  be  greatly  facilitated  if  the  DoD  would  adopt  a  standard 
terminology  across  the  entire  military  health  system. 
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Appendix  1:  CDC  laboratory  syndrome  definitions 


syndrome  name 

syndromedefinition 

Laboratory  tests 

Gastrointestinal 

ACUTE  infection  of  the  upper  and /  or  lower 
gastrointestinal  (Gl)  tract;  SPECIFIC  diagnosis 
of  acute  Gl  distress  such  as  Salmonella 
gastroenteritis;  ACUTE  non-specific  symptoms 
of  Gl  distress  such  as  nausea,  vomiting,  or 
diarrhea;  EXCLUDES  any  chronic  conditions 
such  as  inflammatory  bowel  syndrome. 

Helicobacter  pylori  Culture 

H  PYLORI,  IGM,  IGG,  IGA 
AB 

H.  pylori  IgG,  Abs 

H.  pylori  Stool  Antigen 
Helicobacter  pylori,  IgA 
Helicobacter  pylori,  IgM  Ab 

C  difficile  Toxin  A 

C  difficile,  Toxin  B/Cytotoxin 
Clostridium  difficile  Culture 
Campylobacter  Culture 
Enterohemorrhagic  E  coli 

Cult 

Adenovirus 
(40/41  )/Rotavirus 

Adenovirus  (40/41),  Direct 
EIA 

Norovirus,  RT-PCR 

Rotavirus  Detection  by  EIA 
Cryptosporidium  Smear, 

Stool 

Amebiasis  Antibodies 
Cyclospora  Smear,  Stool 
Giardia  lamblia,  Direct,  EIA 
Giardia,  EIA;  Ova/Parasites 
Stool  Culture 

Stool  Culture,  Yersinia  Only 
Stool  Culture,  Vibrio  Only 
White  Blood  Cells  (WBC), 
Stool 

Occult  Blood,  Stool 
Ova/Parasites  Exam, 

Routine 

Fecal  Reducing  Substances 

Respiratory 

ACUTE  infection  of  the  upper  and /  or  lower 
respiratory  tract  (from  the  oropharynx  to  the 
lungs,  includes  otitis  media);  SPECIFIC 
diagnosis  of  acute  respiratory  tract  infection 
(RTI)  such  as  pneumonia  due  to  parainfluenza 
virus;  ACUTE  non-specific  diagnosis  of  RTI 
such  as  sinusitis,  pharyngitis,  laryngitis 

ACUTE  non-specific  symptoms  of  RTI  such  as 
cough,  stridor,  shortness  of  breath,  throat 
pain;  EXCLUDES  chronic  conditions  such  as 
chronic  bronchitis,  asthma  without  acute 
exacerbation,  chronic  sinusitis,  allergic 
conditions  (Note:  INCLUDE  acute 
exacerbation  of  chronic  illnesses.) 

AFB  Cult/Smear,  Broth, 
Suscep 

AFB  Culture  and  Smear, 
Broth 

Organism  ID,  Mycobacteria 

M  tuberculosis  Detection, 
PCR 

M  tuberculosis,  PCR/Culture 
Mycoplasma  pneumoniae 
Culture 

Mycoplasma  Pneumoniae, 
PCR 

Mycoplasma  pneu.  IgG/IgM 
Abs 

Mycoplasma  pneumoniae, 
IgG  Ab 
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syndromename 

syndromedefinition 

Laboratory  tests 

Mycoplasma  pneumoniae, 
IgM  Ab 

Adenovirus  Group  Ab,  Qn 
Adenovirus  Detection  by 

PCR 

Virus,  Adenovirus  by  DFA 

B  pertussis  Smear,  DFA 

B  pertussis  IgA  Ab,  Quant 

B  pertussis  IgG/M/A  Ab, 
Quant 

B  pertussis,  Nasophar 
Culture 

Bordetella  Para&Pertussis 
PCR 

B  pertussis  IgG  Ab,  Quant 

B  pertussis  IgG/IgM  Ab, 
Quant 

B  pertussis  IgM  Ab,  Quant 
Beta-Hemolytic  Strep,  A 

Only 

Streptococcus  pneumoniae 
Ag 

Beta  Strep  (Group  B) 

Antigen 

Chlamydia  psittaci  Culture 
Chlamydia  Pneumoniae 

PCR 

Haemophilus  influenzae  B 

Ag 

Haemophilus  influenzae  B 

IgG 

Influenza  A  Only  by  Direct 
EIA 

Viral 

Culture,  Rapid,  Influenza 
Influenza  A/B  Ab,  Quant 
Influenza  A  &  B, 
Immunoassay 

Parainfluenza  Virus 

Antibody 

Legionella  Species  Culture 
Legionella  pneumophila  Ur 
Ag 

Legionella  pneumophila  by 
DFA 

Legionella 

pneumophila/Culture 

RSV  Ab,  Quant 

Virus,  RSV  by  DFA 

RSV  by  EIA 

Upper  Respiratory  Culture 
Lower  Respiratory  Culture 
Viral 

Culture,  Rapid,  Respirator 
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syndromename 

syndromedefinition 

Laboratory  tests 

Brucella  abortus  IgG,  EIA 
Brucella  abortus  IgM,  EIA 
Diphtheria  Antitoxoid  Ab 

Q  Fever  Antibodies,  IgG 
RESPIRATORY 

INFECTION  PROF  A 
RESPIRATORY 

INFECTION  PROF  B 
Respiratory  Infection  Prof  D 
Fungal  Antibodies,  Quant 
Histoplasma  Abs,  Qn,  DID 

Fever 

ACUTE  potentially  febrile  illness  of  origin  not 
specified;  INCLUDES  fever  and  septicemia 
not  otherwise  specified;  INCLUDES 
unspecified  viral  illness  even  though  unknown 
if  fever  is  present;  EXCLUDE  entry  in  this 
syndrome  category  if  more  specific  diagnostic 
code  is  present  allowing  same  patient  visit  to 
be  categorized  as  respiratory,  neurological  or 
gastrointestinal  illness  syndrome. 

Aerobic  Bacterial  Culture 
Anaerobic  Culture 

Anaerobic  and  Aerobic 
Culture 

Blood  Culture,  Routine 

Viral  Culture,  General 
Anaerobic/ Aerobic/Gram's 
Stain 

Febrile  Agglutinin  Panel 
Malarial  Smear 

Ehrlichia  Ab  Panel 

Human  Gran.  Ehrlichiosis 
(igQ) 

Human  Monocytic  Ehrlich- 
PCR 

E.  chaffeenis-HME 
(Monocytic) 

Ehrlichia  Detection  by  PCR 
Human  Granulocytic 
Ehrlich-HGE 

Gram's  Stain 

Aerobe  ID  &  Suscept 
Anaerobe  Identification  Only 
MRSA  Culture  Only 

MRSA  Culture/Susceptibility 

Vancomycin-Resist 

Enterococcus 

MIC/Min  Bactericidal  Cone 

Neurological 

ACUTE  neurological  infection  of  the  central 
nervous  system  (CNS);  SPECIFIC  diagnosis 
of  acute  CNS  infection  such  as  pneumoccocal 
meningitis,  viral  encephailitis;  ACUTE  non¬ 
specific  diagnosis  of  CNS  infection  such  as 
meningitis  not  otherwise  specified  (NOS), 
encephailitis  NOS,  encephalopathy  NOS 
ACUTE  non-specific  symptoms  of  CNS 
infection  such  as  meningismus,  delerium; 
EXCLUDES  any  chronic,  hereditary  or 
degenerative  conditions  of  the  CNS  such  as 
obstructive  hydrocephalus,  Parkinson’s, 
Alzheimer's. 

Glucose,  Cerebrospinal 

Fluid 

Protein,  Total,  CSF 

Bacterial  Antigens 

Cell  Count,  CSF 

Calif  Encephalitis  Ab,  IgG 
Calif  Encephalitis  Ab,  IgM 
India  Ink  Preparation 
Cryptococcus  Antibodies, 
Quant 

Cryptococcus  Antigen,  CSF 
Cryptococcus  Antigen, 

Serum 

East  Eq  Encephalitis  Ab, 

IgG 
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syndromedefinition 

Laboratory  tests 

East  Eq  Encephalitis  Ab, 

IgM 

St  Louis  Enceph  V  Ab,  IgG 

St  Louis  Enceph  V  Ab,  IgM 
Western  Equine  Enceph  Ab, 
IgG 

Western  Equine  Enceph  Ab, 
IgM 

West  Nile  Virus 

Antibody, Serum 

West  Nile  Virus,  RT-PCR 
West  Nile  Virus  Antibody, 
CSF 

Arboviral  Encephalitis  Ab, 

IgM 

Arboviral  Encephalitis  Ab, 

IgG 

JC/BK  Virus  DNA  PCR 
Enterovirus  RT-PCR 

Lyme  Ab/Total 
Immunoglobulins 
LYME/SYPHILIS  AB  DIFF 
PROFILE 

Lyme  Ab,  Total/IgM 
Responses 

Lyme  Ab/Western  Blot 

Reflex 

Lyme 

Disease(B.Burgdorferi)PCR 
Lyme  PCR,  Borrelia 
burgdorferi 

Lyme,  IgM,  Early 

Test/Reflex 

Lyme,  Total  Ab  Test/Reflex 
Lyme,  Western  Blot,  Serum 
Lyme,  Western  Blot,  Syn 
Fluid 

Botulism-like 

ACUTE  condition  that  may  represent 
exposure  to  botulinum  toxin;  ACUTE  paralytic 
conditions  consistent  with  botulism:  cranial 
nerve  VI  (lateral  rectus)  palsy,  ptosis,  dilated 
pupils,  decreased  gag  reflex,  media  rectus 
palsy.  ACUTE  descending  motor  paralysis 
(including  muscles  of  respiration);  ACUTE 
symptoms  consistent  with  botulism:  diplopia, 
dry  mouth,  dysphagia,  difficulty  focusing  to  a 
near  point. 

None 

Hemorrhagic  Illness 

SPECIFIC  diagnosis  of  any  virus  that  causes 
viral  hemorrhagic  fever  (VHF):  yellow  fever, 
dengue,  Rift  Valley  fever,  Crimean-Congo  HF, 
Kyasanur  Forest  disease,  Omsk  HF,  Hantaan, 
Junin,  Machupo,  Lassa,  Marburg,  Ebola; 
ACUTE  condition  with  multiple  organ 
involvement  that  may  be  consistent  with 

None 
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syndromedefinition 

Laboratory  tests 

exposure  to  any  virus  that  causes  VHF; 

ACUTE  blood  abnormalities  consistent  with 
VHF:  leukopenia,  neutropenia, 
thrombocytopenia,  decreased  clotting  factors, 
albuminuria. 

Lymphadenitis 

ACUTE  regional  lymph  node  swelling  and/  or 
infection  (painful  bubo-  particularly  in  groin, 
axilla  or  neck) 

Cytomegalovirus  (CMV) 
Culture 

Virus,  Cytomegalovirus  by 
DFA 

Cytomegalovirus  Quant. 

PCR 

Cytomegalovirus  (CMV)  Ab, 
IgM 

CMV  PCR  Southern  Blot 
CMV  DNA  Probe,  Paraffin 
CMV  PCR  Detect. ,Amniotic 
Fluid 

Viral  Culture,  Rapid,  CMV 
Mono  Qual  W/Rflx  Qn 
Mononucleosis  Test,  Qual 
EBV  Ab  VCA,  IgG 

EBV  Early  Antigen  Ab  Prof, 
Qn 

EBV  Early  Antigen  Ab,  IgG 
EBV  Ab  VCA,  IgM 
Epstein-Barr  Virus  Real 

Time 

Epstein-Barr  DNA  PCR 

Real  Time 

Epstein-Barr  Virus,  DNA 
Probe 

Mumps  Antibodies,  IgG 
Tularemia  Agglutinins 
Toxoplasma  gondii  Ab,  IgG, 
Qn 

Toxoplasma  Gondii  PCR 
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Localized 

Cutaneous  Lesion  / 
Rash 

SPECIFIC  diagnosis  of  localized  cutaneous 
lesion/  ulcer  consistent  with  cutaneous 
anthrax  or  tularemia;  ACUTE  localized  edema 
and/  or  cutaneous  lesion/  vesicle,  ulcer, 
eschar  that  may  be  consistent  with  cutaneous 
anthrax  or  tularemia;  INCLUDES  insect  bites; 
EXCLUDES  any  lesion  disseminated  over  the 
body  or  generalized  rash;  EXCLUDES 
diabetic  ulcer  and  ulcer  associated  with 
peripheral  vascular  disease. 

ACUTE  condition  that  may  present  as 
consistent  with  smallpox  (macules,  papules, 
vesicles  predominantly  of  face/arms/legs); 
SPECIFIC  diagnosis  of  acute  rash  such  as 
chicken  pox  in  person  >  XX  years  of  age 
(base  age  cut-off  on  data  interpretation)  or 
smallpox;  ACUTE  non-specific  diagnosis  of 
rash  compatible  with  infectious  disease,  such 
as  viral  exanthem;  EXCLUDES  allergic  or 
inflammatory  skin  conditions  such  as  contact 
or  seborrheaic  dermatitis,  rosacea; 

EXCLUDES  rash  NOS,  rash  due  to  poison  ivy, 
sunburn,  and  eczema. 

Rocky  Mtn  Spotted  Fev, 

IgG,  Qn 

Rocky  Mtn  Spotted  Fever, 
IgM 

Rubella  Antibodies,  IgM 
Rubella  Antibodies,  IgG 
Human  Papillomavirus, 
Biopsy 

Human  Papillomavirus, 

PCR 

Viral  Culture, Rapid, Varicella 
Virus,  Varicella  Zoster  by 
DFA 

Varicella-Zoster  Ab,  IgM 
Varicella-Zoster  V  Ab,  IgG 
VZV  Real  Time  PCR 
Parvovirus  B1 9  PCR  Amn. 

FI  Det 

Parvovirus  B19  PCR 
Detection 

RASH  PROFILE  B 

Rash  Profile  A 

Measles/Mumps/Rubella 

Immunity 

Rubeola  Antibodies,  IgG 
HHV-6,  IgG  Antibodies, 
Quant 

Viral  Culture, Rapid, Lesion 
Virus,  HSV  by  DFA 

HSV  1/2  PCR 

HSV  Culture  and  Typing 

HSV  I/ll,  IgG/Rfx  Type  II  IgG 
HSV  Type  2-Specific  Ab, 

IgG 

HSV,  IgM  I/ll  Combination 
Herpes  Simplex  Virus,  DNA 
HSV  Culture  Without  Typing 
Herpes  Simplex  Virus  I/ll, 

IgG 

Dermatophyte  Only,  Culture 

Organism  ID,  Bacteria 
Organism  Identification, 
Yeast 

Genital  Culture,  Routine 

Urine  Culture, 
Comprehensive 

Reference  Bacterial  Culture 
ID 

Susceptibility,  Aer  & 

Anaerob 

Parasite  Identification 

Fungus  (Mycology)  Culture 
Fungus  Stain 

Specific  Infection 

ACUTE  infection  of  known  cause  not  covered 
in  other  syndrome  groups,  usually  has  more 
generalized  symptoms  (i.e.,  not  just 
respiratory  or  gastrointestinal);  INCLUDES 
septicemia  from  known  bacteria;  INCLUDES 
other  febrile  illnesses  such  as  scarlet  fever. 
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syndromename 

syndromedefinition 

Laboratory  tests 

Fungus  Culture  With  Stain 
Candida  Antibodies,  Qual 
Anti-DNase  B  Strep 
Antibodies 

Tetanus/Diphtheria  Ab 
Chlamydia  Antibodies,  IgG 
Chlamydia 

trach.Swab/Urine,PCR 
Chlamydia  trachomatis  Ab, 
IgM 

Chlamydia  trachomatis 
Culture 

Echinococcus  Antibody 

Hep  A  Ab,  IgM 

Hep  A  Ab,  Total 

Hep  B  Core  Ab,  IgM 

Hep  B  Core  Ab,  Tot 

Hep  B  Surface  Ab 

Hep  B  Surface  Ag 

Hep  Be  Ab 

Hep  Be  Ag 

Hep  C  Virus  Ab 

HAV/HBV  (Profile  VII) 

HBV  Core  Ab,  IgG/IgM  Diff 
HBV/HCV  (Profile  VIII) 

HCV  QuantaSure  Plus 
(Serial) 

HCV  QuantaSure  Plus(Non- 
Graph) 

HCV  RNA  by  PCR,  Qn  Rfx 
Geno 

HCV  RNA,  PCR,  Qualitative 
Hepatitis  A  (Prof  V) 

Hepatitis  B  Virus  (Profile  VI) 
Hepatitis  C  Virus 

Genotyping 

Hepatitis  Follow-Up  (Prof  II) 
Hepatitis  Panel  (4) 

Hepatitis  Pt  Mgmnt  (Prof  III) 
Hepatitis,  Diagnostic  (Prof  1) 
NGI  HBV  SuperQuant 

NGI  HBV  UltraQual 

NGI  HCV  QuantaSure 

NGI  HCV  SuperQuant 

NGI  HCV  UltraQual 

HAV/HBV  Immune  Status 
(Pro  IV) 

HBV  DNA,  Qualitative  PCR 
HBV  Follow-Up  (Profile  XII) 
HBV  Prevaccination  (Profile 
X) 

HBV  Vaccine  Follow-Up 
(Pro  XI) 

HCV  RNA  Det  Ql  Rfx  Gen 
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syndromename 

syndromedefinition 

Laboratory  tests 

HCV  RNA  by  PCR,  Qn  Rfx 
Geno 

HCV  RNA,  PCR,  Ql  (Quant 
Rflx) 

Hepatitis  B,  Prenatal  (Prof 

X 

Strep  pneumo  IgG  Ab  (6 
Sero) 

Strep  pneumo  IgG  Ab  (7 
Sero.) 

Strep,  pneumo. IgG  Ab  (4 
Sero.) 

Strep. pneumo. IgG  Ab  (14 
Sero) 

Prenat  Infect  Dis  Ab,  IgG, 

Qn 

Prenat  Infect  Dis  Ab,  IgM, 

Qn 

HTLV-I/II  Antibodies,  Qual 

Ureaplasma/Mycoplasma 

hominis 

Severe  Illness  or 
Death  potentially 
due  to  infectious 
disease 

ACUTE  onset  of  shock  or  coma  from 
potentially  infectious  causes;  EXCLUDES 
shock  from  trauma;  INCLUDES  SUDDEN 
death,  death  in  emergency  room,  intrauterine 
deaths,  fetal  death,  spontaneous  abortion,  and 
still  births;  EXCLUDES  induced  fetal 
abortions,  deaths  of  unknown  cause,  and 
unattended  deaths. 

None 
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Appendix  2:  MTFs  with  complete  data 


Facility  Name 

DMIS 

ID 

Branch  of 
service 

Facility 

Type 

US 

Flag? 

Avg.  daily 
tests 

1 1TH  MED  GRP-BOLLING 

0413 

F 

CLINIC 

Y 

16 

14th  MED  GRP-COLUMBUS 

0074 

F 

CLINIC 

Y 

3 

15th  MED  GRP-HICKAM 

0287 

F 

CLINIC 

Y 

10 

16th  MED  GRP-HURLBURT 

7139 

F 

CLINIC 

Y 

14 

FIELD 

18th  MED  GRP-KADENA  AB 

0804 

F 

CLINIC 

N 

18 

1st  MED  GRP-LANGLEY 

0120 

F 

HOSP 

Y 

74 

20th  MED  GRP-SHAW 

0101 

F 

HOSP 

Y 

16 

27th  MED  GRP-CANNON 

0085 

F 

CLINIC 

Y 

15 

30th  MED  GRP-VANDENBERG 

0018 

F 

CLINIC 

Y 

11 

319th  MED  GRP-GRAND  FORKS 

0093 

F 

CLINIC 

Y 

7 

31st  MED  GRP-AVIANO 

0808 

F 

HOSP 

N 

12 

325th  MED  GRP-TYNDALL 

0043 

F 

CLINIC 

Y 

14 

341st  MED  GRP-MALMSTROM 

0077 

F 

CLINIC 

Y 

7 

354th  MED  GRP-EIELSON 

0203 

F 

CLINIC 

Y 

12 

366th  MED  GRP-MOUNTAIN 

0053 

F 

HOSP 

Y 

14 

HOME 

374th  MED  GRP-YOKOTA  AB 

0640 

F 

HOSP 

N 

25 

3rd  MED  GRP-ELMENDORF 

0006 

F 

HOSP 

Y 

40 

401  EABG/SG-TUZLA  AB 

6704 

F 

ADMIN 

N 

1 

422  ABS  MED  FLT- 

0653 

F 

CLINIC 

N 

2 

CROUGHTON 

423RD  ABS  OL-A-RAF 

0814 

F 

CLINIC 

N 

2 

UPWOOD 

437th  MED  GRP-CHARLESTON 

0356 

F 

CLINIC 

Y 

18 

469th  MED  FLT-RHEIN  MAIN 

0800 

F 

CLINIC 

N 

4 

470  MED  FLT-GEILENKIRCHEN 

0799 

F 

CLINIC 

N 

2 

48th  MED  GRP-LAKENHEATH 

0633 

F 

HOSP 

N 

35 

4th  MED  GRP-SEYMOUR 

0090 

F 

CLINIC 

Y 

17 

JOHNSON 

51st  MED  GRP-OSAN  AB 

0638 

F 

HOSP 

N 

10 

52nd  MED  GROUP- 

0805 

F 

HOSP 

N 

19 

SPANGDAHLEM 

56th  MED  GRP-LUKE 

0009 

F 

HOSP 

Y 

48 

5th  MED  GRP-MINOT 

0094 

F 

CLINIC 

Y 

10 

601st  MED  SQUAD-SEMBACH 

0801 

F 

INACT 

N 

22 

60th  MED  GRP-TRAVIS 

0014 

F 

HOSP 

Y 

79 

61st  MED  SQUAD-LOS 

0248 

F 

CLINIC 

Y 

6 

ANGELES 

66th  MED  GRP-HANSCOM 

0310 

F 

CLINIC 

Y 

11 

6th  MED  GRP-MACDILL 

0045 

F 

HOSP 

Y 

76 

7020th  ABG  CLINIC-FAIRFORD 

0815 

F 

CLINIC 

N 

0 

74th  MED  GRP-WRIGHT- 

0095 

F 

HOSP 

Y 

115 

PATTERSON 

75th  MED  GRP-HILL 

0119 

F 

CLINIC 

Y 

37 

7th  MED  GRP-DYESS 

0112 

F 

CLINIC 

Y 

14 

82nd  MED  GRP-SHEPPARD 

0113 

F 

HOSP 

Y 

35 

89th  MED  GRP-ANDREWS 

0066 

F 

HOSP 

Y 

99 
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Facility  Name 

DMIS 

ID 

Branch  of 
service 

Facility 

Type 

US 

Flag? 

Avg.  daily 
tests 

8th  MED  GRP-KUNSAN  AB 

0637 

F 

CLINIC 

N 

2 

90th  MED  GRP-F.E.  WARREN 

0129 

F 

CLINIC 

Y 

19 

96th  MED  GRP-EGLIN 

0042 

F 

HOSP 

Y 

65 

9th  MED  GRP-BEALE 

0015 

F 

CLINIC 

Y 

14 

AHC  BAUMHOLDER 

1007 

A 

INACT 

N 

9 

AHC  BRUSSELS 

8977 

A 

CLINIC 

N 

0 

AHC  BUTZBACH 

8996 

A 

CLINIC 

N 

2 

AHC  COLEMAN 

7152 

A 

CLINIC 

N 

1 

AHC  DARMSTADT 

8998 

A 

CLINIC 

N 

5 

AHC  DEXHEIM 

8992 

A 

CLINIC 

N 

1 

AHC  FRIEDBERG 

1135 

A 

CLINIC 

N 

1 

AHC  FT.  STORY 

0464 

A 

CLINIC 

Y 

1 

AHC  GIEBELSTADT 

1235 

A 

CLINIC 

N 

1 

AHC  GRAFENWOEHR 

1016 

A 

CLINIC 

N 

1 

AHC  HANAU 

8995 

A 

CLINIC 

N 

7 

AHC  HOHENFELS 

1019 

A 

CLINIC 

N 

2 

AHC  ILLESHEIM 

1014 

A 

CLINIC 

N 

1 

AHC  KAISERSLAUTERN 

1128 

A 

CLINIC 

N 

3 

AHC  KATTERBACH 

1015 

A 

CLINIC 

N 

2 

AHC  KELLY-STUTTGART 

1233 

A 

INACT 

N 

5 

AHC  KITZINGEN 

1127 

A 

CLINIC 

N 

2 

AHC  LIVORNO 

1154 

A 

CLINIC 

N 

1 

AHC  MANNHEIM 

1003 

A 

CLINIC 

N 

9 

AHC  MCAFEE-WHITE  SANDS 

MSL RAN 

0327 

A 

CLINIC 

Y 

1 

AHC  SCHWEINFURT 

1124 

A 

CLINIC 

N 

4 

AHC  SHAPE 

0614 

A 

CLINIC 

N 

4 

AHC  VICENZA 

0611 

A 

CLINIC 

N 

6 

AHC  VILSECK 

1017 

A 

CLINIC 

N 

6 

AHC  WIESBADEN 

1147 

A 

CLINIC 

N 

10 

ANDREW  RADER  AHC-FT. 

MYER 

0390 

A 

CLINIC 

Y 

16 

BASSETT  ACH-FT. 

WAINWRIGHT 

0005 

A 

HOSP 

Y 

30 

BAYNE-JONES  ACH-FT.  POLK 

0064 

A 

HOSP 

Y 

28 

BLANCHFIELD  ACH-FT. 
CAMPBELL 

0060 

A 

HOSP 

Y 

93 

BMA  NAVCAMS  EASTPAC 

0284 

N 

CLINIC 

Y 

1 

BMC  ALBANY 

0275 

N 

CLINIC 

Y 

1 

BMC  ANDREWS  AFB 

0522 

N 

CLINIC 

Y 

1 

BMC  ATHENS 

0276 

N 

CLINIC 

Y 

0 

BMC  BARSTOW 

0209 

N 

CLINIC 

Y 

1 

BMC  CAMP  BUSH/COURTNEY 

7032 

N 

CLINIC 

N 

4 

BMC  CAMP  DELMAR  MCB 

1657 

N 

CLINIC 

Y 

0 

BMC  CAMP  GEIGER  MCB 

1662 

N 

CLINIC 

Y 

9 

BMC  CAMP  HANSEN 

7033 

N 

CLINIC 

N 

2 

BMC  CAMP  JOHNSON  MCB 

1663 

N 

CLINIC 

Y 

4 

BMC  CAMP  KINSER 

1269 

N 

CLINIC 

N 

3 

BMC  CAMP  SCHWAB-OKINAWA 

7107 

N 

CLINIC 

N 

1 

BMC  CAPODICHINO 

1153 

N 

CLINIC 

N 

3 

BMC  CHESAPEAKE 

0519 

N 

CLINIC 

Y 

1 
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Facility  Name 

DMIS 

ID 

Branch  of 
service 

Facility 

Type 

US 

Flag? 

Avg.  daily 
tests 

BMC  COMFLEACT  SASEBO 

0852 

N 

CLINIC 

N 

2 

BMC  CORCEN  MCB 

1975 

N 

CLINIC 

Y 

0 

BMC  CORONADO 

0233 

N 

CLINIC 

Y 

0 

BMC  DAHLGREN 

0386 

N 

CLINIC 

Y 

1 

BMC  DAM  NECK 

0382 

N 

CLINIC 

Y 

8 

BMC  EDSON  RANGE  ANNEX 

0210 

N 

CLINIC 

Y 

12 

BMC  EL  CENTRO 

0239 

N 

CLINIC 

Y 

1 

BMC  EVANS-CAMP  FOSTER 

0862 

N 

CLINIC 

N 

4 

BMC  FRENCH  CREEK  MCB 

1995 

N 

CLINIC 

Y 

5 

BMC  GAETA 

0874 

N 

CLINIC 

N 

1 

BMC  INDIAN  HEAD 

0301 

N 

CLINIC 

Y 

2 

BMC  IWAKUNI 

0625 

N 

CLINIC 

N 

8 

BMC  KEY  WEST 

0041 

N 

INACT 

Y 

2 

BMC  LITTLE  CREEK 

0378 

N 

CLINIC 

Y 

51 

BMC  MARIETTA 

0277 

N 

CLINIC 

Y 

1 

BMC  MAYPORT 

0405 

N 

CLINIC 

Y 

14 

BMC  MCAS  BEAUFORT 

0360 

N 

CLINIC 

Y 

2 

BMC  MCAS  FUTENMA 

0861 

N 

CLINIC 

N 

1 

BMC  MCAS  KANEOHE  BAY 

0285 

N 

CLINIC 

Y 

8 

BMC  MCAS  MIRAMAR 

0232 

N 

CLINIC 

Y 

13 

BMC  MCAS  NEW  RIVER 

0333 

N 

CLINIC 

Y 

3 

BMC  MCB  CAMP  H.M.  SMITH 

1987 

N 

CLINIC 

Y 

1 

BMC  MCB  CAMP  PENDLETON 

0208 

N 

CLINIC 

Y 

1 

BMC  MCRD  PARRIS  ISLAND 

0358 

N 

CLINIC 

Y 

30 

BMC  MCRD  SAN  DIEGO 

0230 

N 

CLINIC 

Y 

16 

BMC  MECHANICSBURG 

0348 

N 

CLINIC 

Y 

0 

BMC  MERIDIAN 

0317 

N 

CLINIC 

Y 

3 

BMC  MILTON  WHITING  FIELD 

0261 

N 

CLINIC 

Y 

5 

BMC  NAF  ATSUGI 

0853 

N 

CLINIC 

N 

9 

BMC  NAS  JACKSONVILLE 

0266 

N 

CLINIC 

Y 

4 

BMC  NAS  MEMPHIS 

0361 

N 

INACT 

Y 

2 

BMC  NAS  NORTH  ISLAND 

0231 

N 

CLINIC 

Y 

7 

BMC  NAS  PENSACOLA 

0260 

N 

CLINIC 

Y 

6 

BMC  NAS  POINT  MUGU 

0217 

N 

CLINIC 

Y 

1 

BMC  NATTC  PENSACOLA 

0262 

N 

CLINIC 

Y 

12 

BMC  NAVCOASTSYSC 

PANAMA  CITY 

0265 

N 

CLINIC 

Y 

1 

BMC  NAVSTA  SAN  DIEGO 

0234 

N 

INACT 

Y 

7 

BMC  NAVSUPPO  LA 
MADDALENA 

0855 

N 

CLINIC 

N 

1 

BMC  NAVWPNCEN  CHINA 

LAKE 

0212 

N 

CLINIC 

Y 

13 

BMC  NAVWPNSFAC  ST. 
MAWGAN 

1179 

N 

CLINIC 

N 

0 

BMC  NSA  BAHRAIN 

1170 

N 

CLINIC 

N 

3 

BMC  NSY  NORFOLK 

0380 

N 

CLINIC 

Y 

0 

BMC  NTC  GREAT  LAKES 

1959 

N 

CLINIC 

Y 

8 

BMC  NTC  SAN  DIEGO 

0407 

N 

CLINIC 

Y 

13 

BMC  NTTC  PENSACOLA 

0513 

N 

CLINIC 

Y 

5 

BMC  OCEANA 

0387 

N 

CLINIC 

Y 

20 

BMC  SAN  ONOFRE  MCB 

1659 

N 

CLINIC 

Y 

2 
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Facility  Name 

DMIS 

ID 

Branch  of 
service 

Facility 

Type 

US 

Flag? 

Avg.  daily 
tests 

BMC  SUGAR  GROVE 

0404 

N 

CLINIC 

Y 

0 

BMC  WILLOW  GROVE 

0347 

N 

CLINIC 

Y 

1 

BMC  YORKTOWN 

0381 

N 

CLINIC 

Y 

1 

BMC  YUMA 

0269 

N 

CLINIC 

Y 

5 

DEWITT  ACH-FT.  BELVOIR 

0123 

A 

HOSP 

Y 

57 

DILORENZO  TRICARE  HLTH 

CLN  ARL 

7298 

A 

CLINIC 

Y 

11 

DSCPL  BRKS  HLTH  CLN-FT 
LEAVENW 

1530 

A 

CLINIC 

Y 

0 

DUGWAY  PROVING  GROUND 
AHC 

0371 

A 

CLINIC 

Y 

0 

DUNHAM  AHC-CARLISLE 
BARRACKS 

0352 

A 

CLINIC 

Y 

17 

FAMILY  HEALTH  CENTER 
FAIRFAX 

6200 

A 

CLINIC 

Y 

25 

FAMILY  HEALTH  CENTER 
WOODBRIDG 

6201 

A 

CLINIC 

Y 

45 

FOX  AHC-REDSTONE 

ARSENAL 

0001 

A 

CLINIC 

Y 

7 

GUTHRIE  AHC-FT.  DRUM 

0330 

A 

CLINIC 

Y 

37 

HEIDELBERG  MEDDAC 

0606 

A 

HOSP 

N 

21 

KENNER  AHC-FT.  LEE 

0122 

A 

CLINIC 

Y 

15 

KIMBROUGH  AMB  CAR  CEN-FT 
MEADE 

0069 

A 

CLINIC 

Y 

28 

KIRK  AHC-ABERDEEN  PRVNG 

GD 

0308 

A 

CLINIC 

Y 

15 

LA  POINTE  HEALTH  CLINIC 

7307 

A 

CLINIC 

Y 

6 

LANDSTUHL  REGIONAL 

MEDCEN 

0607 

A 

HOSP 

N 

48 

LYSTER  ACH-FT.  RUCKER 

0003 

A 

HOSP 

Y 

21 

MARTIN  ACH-FT.  BENNING 

0048 

A 

HOSP 

Y 

131 

MCDONALD  ACH-FT.  EUSTIS 

0121 

A 

HOSP 

Y 

70 

MENWITH  HILL  MEDICAL 

CENTER 

7234 

F 

CLINIC 

N 

0 

MONCRIEF  ACH-FT.  JACKSON 

0105 

A 

HOSP 

Y 

60 

MONROE  AHC-FT.  MONROE 

0372 

A 

CLINIC 

Y 

2 

MONTEREY  AHC 

0247 

A 

CLINIC 

Y 

1 

MUNSON  AHC-FT. 
LEAVENWORTH 

0058 

A 

CLINIC 

Y 

19 

NACC  KINGS  BAY 

0337 

N 

CLINIC 

Y 

5 

NAS  NORFOLK  NRMC/BC 

0377 

N 

CLINIC 

Y 

28 

NH  BEAUFORT 

0104 

N 

HOSP 

Y 

37 

NH  CAMP  LEJEUNE 

0091 

N 

HOSP 

Y 

120 

NH  CAMP  PENDLETON 

0024 

N 

HOSP 

Y 

48 

NH  CHARLESTON 

0103 

N 

HOSP 

Y 

13 

NH  CHERRY  POINT 

0092 

N 

HOSP 

Y 

60 

NH  GREAT  LAKES 

0056 

N 

HOSP 

Y 

47 

NH  GUAM-AGANA 

0620 

N 

HOSP 

Y 

25 

NH  JACKSONVILLE 

0039 

N 

HOSP 

Y 

36 

NH  KEFLAVIK 

0623 

N 

HOSP 

N 

4 

NH  NAPLES 

0617 

N 

HOSP 

N 

20 

NH  OKINAWA 

0621 

N 

HOSP 

N 

30 
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Facility  Name 

DMIS 

ID 

Branch  of 
service 

Facility 

Type 

US 

Flag? 

Avg.  daily 
tests 

NH  PENSACOLA 

0038 

N 

HOSP 

Y 

92 

NH  TWENTYNINE  PALMS 

0030 

N 

HOSP 

Y 

26 

NH  YOKOSUKA 

0622 

N 

HOSP 

N 

38 

NMC  PORTSMOUTH 

0124 

N 

HOSP 

Y 

80 

NMC  SAN  DIEGO 

0029 

N 

HOSP 

Y 

139 

NMOL  ANNAPOLIS 

0306 

N 

CLINIC 

Y 

13 

NMOL  LONDON 

8931 

N 

CLINIC 

N 

2 

NMOL  PATUXENT  RIVER 

0068 

N 

CLINIC 

Y 

12 

NMOL  PEARL  HARBOR 

0280 

N 

CLINIC 

Y 

28 

NMOL  QUANTICO 

0385 

N 

CLINIC 

Y 

29 

NNMC  BETHESDA 

0067 

N 

HOSP 

Y 

85 

OP  FORCES-NH  GREAT  LAKES 

6308 

N 

ADMIN 

Y 

20 

RECEPTION  STA  TMC-FT. 
BENNING 

1939 

A 

CLINIC 

Y 

1 

REYNOLDS  ACH-FT.  SILL 

0098 

A 

HOSP 

Y 

95 

RICHARDS-GEBAUR  CL- 
KANSAS  CITY 

7297 

A 

CLINIC 

Y 

3 

SCHOFIELD  BARRACKS  AHC 

0437 

A 

CLINIC 

Y 

28 

SGT  BLEAK  TROOP  MED  CLN- 
FT  SIL 

1625 

A 

CLINIC 

Y 

8 

SOUTHCOM  CLINIC 

7239 

A 

CLINIC 

Y 

0 

TMC  CON  (1-2-4)-FT.  JACKSON 

1567 

A 

CLINIC 

Y 

19 

TMC  FT.  RICHARDSON 

0204 

A 

CLINIC 

Y 

1 

TMC-1-FT.  BENNING 

1551 

A 

CLINIC 

Y 

2 

TMC-1-FT.  STEWART 

1562 

A 

CLINIC 

Y 

5 

TMC-1-SCHOF  25th-SCHOFIELD 
BKS 

0534 

A 

CLINIC 

Y 

8 

TMC-2-FT.  BENNING 

1552 

A 

CLINIC 

Y 

2 

TMC-5-FT.  BENNING 

1555 

A 

CLINIC 

Y 

4 

TMC-7-FT.  BENNING 

1557 

A 

CLINIC 

Y 

3 

TOOELE  ARMY  DEPOT  AHC 

0443 

A 

CLINIC 

Y 

0 

TRICARE  OUTPATIENT 
CHESAPEAKE 

6221 

N 

CLINIC 

Y 

16 

TRICARE  OUTPATIENT  CL  VA 
BEACH 

6214 

N 

CLINIC 

Y 

33 

TRICARE  OUTPATIENT-CHULA 
VISTA 

6215 

N 

CLINIC 

Y 

19 

TRICARE  OUTPATIENT- 
CLAIRMONT 

6207 

N 

CLINIC 

Y 

18 

TRICARE  OUTPATIENT- 
OCEANSIDE 

6216 

N 

CLINIC 

Y 

6 

TRIPLER  AMC-FT  SHAFTER 

0052 

A 

HOSP 

Y 

101 

TUTTLE  AHC-HUNTER  AB 

0272 

A 

CLINIC 

Y 

15 

USA  MEDDAC-CAMP  ZAMA 

0610 

A 

CLINIC 

N 

6 

WALTER  REED  AMC- 
WASHINGTON  DC 

0037 

A 

HOSP 

Y 

99 

WILLIAM  BEAUMONT  AMC-FT. 
BLISS 

0108 

A 

HOSP 

Y 

48 

WINDER  FPC-FT.  BENNING 

1316 

A 

CLINIC 

Y 

17 

WINN  ACH-FT.  STEWART 

0049 

A 

HOSP 

Y 

103 

WUERZBURG  MEDDAC 

0609 

A 

HOSP 

N 

16 
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Appendix  3:  DoD-ESSENCE  ICD-9  Syndrome  Definitions 


Syndrome=Bot-like 


icd9  Dx 

005.1  Botulism 

038.2  Septicemia,  pneumococcal 

344.00  Quadriplegia,  Unspec. 

344.04  Quadriple/Quadripa . C5-C7 

344 . 09  Quadriplegia/Quadriparesis 

344.2  Diplegia  of  upper  limbs 

344.89  Paralytic  Syndrome,  Other 

344.9  Paralysis 

351.8  Neuralgia,  Facial 

351.9  Facial  Nerve  Disorder,  Unspec 

352.6  Cranial  Nerve  Palsies,  Mult. 

352.9  Cranial  Nerve  Disorder,  Unspec 

368.2  Diplopia 

374.30  Ptosis  of  Eyelid,  Unspec. 

374.31  Paralytic  Ptosis 

378.51  Nerve  Palsy,  3rd  Or  Oculomotor,  Partial 

378.52  Nerve  Palsy,  3rd  Or  Oculomotor,  Total 

784.3  Aphasia 


Syndrome: 

II 

CD 

< 

CD 

h5 

icd9 

Dx 

038 . 8 

Septicemia  NEC 

038 . 9 

Septicemia  NOS 

066.1 

Fever,  tick-borne 

066.3 

Fever,  mosquito-borne 

NEC 

066.8 

Disease,  arthpd-borne 

viral 

NEC 

066.9 

Disease,  arthpd-borne 

viral 

NOS 

078.2 

Sweating  fever 

079.89 

Infection,  viral  NEC 

079.99 

Infection,  viral  NOS 

780 .31 

Convulsions,  febrile 

780 . 6 

Fever 

790.7 

Bacteremia 

790 . 8 

Viremia  NOS 

795.39 

NONSP  POSITIVE  CULT  NEC 

Syndrome=GI 
icd9  Dx 

001.0  Cholera  d/t  Vibrio  cholerae 

001.1  Cholera  d/t  Vibrio  cholerae  el  tor 

001.9  Cholera  NOS 

003.0  Gastroenteritis,  salmonella 

003.8  Infection,  salmonella  NEC 
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003.9  Infection,  salmonella  NOS 

004.0  Dysentery,  Shigella  dysenteriae 

004.1  Dysentery,  Shigella  flexneri 

004.2  Dysentery,  Shigella  boydii 

004.3  Dysentery,  Shigella  sonnei 

004.8  Infection,  Shigella  NEC 

004.9  Shigellosis  NOS 

005.0  Poisoning,  food,  staphylococcal 

005.2  Pois,  food,  d/t  C.  perfringens 

005.3  Pois,  food,  d/t  Clostridia  NEC 

005.4  Pois,  food,  d/t  v.  parahaemolyt 

005.81  Pois,  food,  d/t  Vibrio  vulnificus 

005.89  Poisoning,  food,  bacterial  NEC 

005.9  Poisoning,  food  NOS 

007.5  CYCLOSPORIASIS 

008.00  Enteritis,  E.  coli  NOS 

008.01  Enteritis,  enteropathogenic  E.  coli 

008.02  Enteritis,  enterotoxigenic  E.  coli 

008.03  Enteritis,  enteroinvasive  E.  coli 

008.04  Enteritis,  enterohemorrhagic  E.coli 

008.09  Enteritis,  E.  coli  NEC 

008.1  Enteritis,  Arizona  group 

008.2  Enteritis,  Aerobacter  aerogenes 

008.3  Enteritis,  Proteus 

008.41  Enteritis,  staphylococcus 

008.43  Enteritis,  Campylobacter 

008.44  Enteritis,  Yersinia  enterocolitica 

008.45  Enteritis,  Clostridium  difficile 

008.46  Enteritis,  anaerobic  NEC 

008.47  Enteritis,  gram-negative  NEC 

008.49  Enteritis,  bacterial  NEC 

008.5  Enteritis,  bacterial  NOS 

008.61  Enteritis  d/t  rotavirus 

008.62  Enteritis  d/t  adenovirus 

008.63  Enteritis  d/t  Norwalk  virus 

008.64  Enteritis  d/t  small  round  virus  NEC 

008.65  Enteritis  d/t  calcivirus 

008.66  Enteritis  d/t  astrovirus 

008.67  Enteritis  d/t  enterovirus  NEC 

008.69  Enteritis  d/t  virus  NEC 

008.8  Enteritis,  viral  NOS 

009.0  Enteritis,  infectious  NOS 

009.1  Enteritis  presumed  infct  origin 

009.2  Diarrhea,  infectious 

009.3  Diarrhea,  presumed  infct  origin 

021.1  Tularemia,  enteric 

022.2  Anthrax,  gastrointestinal 

078.82  Syndrome,  epidemic  vomiting 

535.00  Gastritis,  acute  w/o  hemorrhage 

535.01  Gastritis,  acute  w/hemorrhage 

535.40  Gastritis  NEC  w/o  hemorrhage 

535.41  Gastritis  NEC  w/hemorrhage 

535.50  Gastritis  NOS  w/o  hemorrhage 

535.51  Gastritis  NOS  w/hemorrhage 

535.60  Duodenitis  w/o  hemorrhage 

535.61  Duodenitis  w/hemorrhage 

536.2  Vomiting,  persistent 
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555.0  Enteritis,  regional  small  intestine 

555.1  Enteritis,  regional  large  intestine 

555.2  Enteritis,  regional  both  intestines 

558.2  Gastroenteritis/colitis,  toxic 

558.9  Gastroenteritis,  noninfct  NEC 

569.9  Disorder,  intestinal  NOS 

787.01  Nausea  with  vomiting 

787.02  Nausea  alone 

787.03  Vomiting  alone 

787.3  Flatulence/eructation/gas  pain 

787.91  Diarrhea  NOS 


Syndr ome=Hemr_i 1 1 
icd9  Dx 

065.0  Hemorrhagic  fever,  Crimean 

065.1  Hemorrhagic  fever,  Omsk 

065.2  Kyasanur  Forest  disease 

065.3  Hemorrhagic  fever,  tick-borne  NEC 

065.4  Hemorrhagic  fever,  mosquito-borne 

065.8  Hemorrhagic  fever,  arthpd-borne  NEC 

065.9  Hemorrhagic  fever,  arthpd-borne  NOS 

077.4  Conjunctivitis,  epidemic  hem 

078.6  Nephrosonephritis ,  hemorrhagic 

078.7  Hemorrhagic  fever,  arenaviral 

084.8  Fever,  blackwater 

100.0  Leptospirosis  icterohemorrhagica 

283.11  Syndrome,  hemolytic-uremic 

286.9  Defect,  coagulation  NEC/NOS 

287.1  Thrombocytopathy 

287.2  Purpura  NOS 

287.3  Thrombocytopenia,  primary 

287.4  Thrombocytopenia,  secondary 

287.5  Thrombocytopenia  NOS 

287.8  Hemorrhagic  condition  NEC 

287.9  Hemorrhagic  condition  NOS 

459.0  Hemorrhage  NOS 

782.7  Ecchymoses,  spontaneous 

790.01  Hematocrit,  Precipitous  Drop 

790.92  Abnormal  blood  coagulation  profile 


Syndr ome=Neuro 
icd9  Dx 

003.21  Meningitis,  salmonella 

036.0  Meningitis,  meningococcal 

036.1  Encephalitis,  meningococcal 

037  Tetanus 

047.0  Meningitis  d/t  Coxsackie  virus 

047.1  Meningitis  d/t  ECHO  virus 

047.8  Meningitis,  viral  NEC 

047.9  Meningitis,  viral  NOS 

048  Disease,  enteroviral  of  CNS  NEC 
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049.0  Choriomeningitis,  lymphocytic 

049.1  Meningitis,  adenovirus 

049.8  Encephalitis,  viral  NEC 

049.9  Encephalitis,  viral  NOS 

052.0  Encephalitis,  postvaricella 

053.0  Herpes  zoster  w/meningitis 

053.10  Herpes  zoster  w/nrv  syst  cmpl  NOS 

054.3  Herpetic  meningoencephalitis 

054.72  Herpes  simplex  meningitis 

055.0  Encephalitis,  postmeasles 

056.00  Rubella  w/neurological  cmpl  NOS 

056.01  Encephalomyelitis  d/t  rubella 

056.09  Rubella  w/neurological  cmpl  NEC 

062.0  Encephalitis,  Japanese 

062.1  Encephalitis,  Western  equine 

062.2  Encephalitis,  Eastern  equine 

062.3  Encephalitis,  St.  Louis 

062.4  Encephalitis,  Australian 

062.5  Encephalitis,  California  virus 

062.8  Encephalitis,  mosquito-borne  NEC 

062.9  Encephalitis,  mosquito-borne  NOS 

063.0  Encephalitis,  Russian  spring-summer 

063.1  Louping  ill 

063.2  Encephalitis,  central  European 

063.8  Encephalitis,  viral,  tick-borne  NEC 

063.9  Encephalitis,  tick-borne  viral  NOS 

064  Encephalitis  arthpd-borne  viral  NEC 

066.4  WEST  NILE  FEVER 

072.1  Mumps  meningitis 

072.2  Mumps  encephalitis 

091.81  Syph  meng,  early,  symp,  acute,  send 

098.82  Meningitis,  gonococcal 

100.81  Meningitis  (aseptic),  leptospiral 

114.2  Coccidioidal  meningitis 

115.01  Histoplasma  capsulatum  meningitis 

115.11  Histoplasma  duboisii  meningitis 

115.91  Histoplasmosis  meningitis 

130.0  Meningoencephalitis,  toxoplasmosis 

320.0  Meningitis,  Hemophilus 

320.1  Meningitis,  pneumococcal 

320.2  Meningitis,  streptococcal 

320.3  Meningitis,  staphylococcal 

320.7  Meng,  in  oth  bctrl  disease  CE 

320.81  Meningitis,  d/t  anaerobic  bacteria 

320.82  Meng,  d/t  gram-negative  bact  NEC 

320.89  Meningitis,  d/t  other  spec  bacteria 

320.9  Meningitis,  d/t  bacteria  NOS 

321.0  Meningitis,  cryptococcal 

321.1  Meningitis  in  other  fungal  disease 

321.2  Meningitis  d/t  viral  diseases  NEC 

321.3  Meningitis  d/t  trypanosomiasis 

321.4  Meningitis  in  sarcoidosis 

321.8  Meng  d/t  oth  nonbact  organism  CE 

322.0  Meningitis,  nonpyogenic 

322.1  Meningitis,  eosinophilic 

322.9  Meningitis  NOS 

323.0  Encephalitis  in  viral  disease  CE 
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323.1 

323.2 

323.4 

323.5 

323.6 

323.7 

323.8 

323.9 
348 . 30 
348 . 39 
781 . 6 


Syndrome 

icd9 

050.0 
050 . 1 
050.2 
050 . 9 
051 . 0 
051 . 1 
051.2 
051 . 9 
052.7 
052.8 
052 . 9 
055.79 
055.8 
055.9 
056.79 
056.8 
056.9 
057 . 0 
057 . 8 
057 . 9 
074 . 3 
082.0 
083.2 
695.0 

695.1 

695.2 
695.89 
695.9 


Syndrome 

icd9 

003.22 
020.3 
020.4 
020 . 5 
021.2 
022 . 1 


Encephalitis  in  rickettsial  dis  CE 
Encephalitis  in  protozoal  dis  CE 
Encephalitis,  oth  d/t  infection  CE 
Encephalitis,  postimmunization 
Encephalitis,  postinf ectious 
Encephalitis,  toxic 
Encephalitis  NEC 
Encephalitis  NOS 
Encephalopathy  NOS 
Encephalopathy  NEC 
Meningismus 


:Rash 

Dx 

Smallpox,  variola  major 
Smallpox,  alastrim 
Smallpox,  modified 
Smallpox  NOS 
Cowpox 

Pseudocowpox 

Dermatitis,  contagious  pustular 
Paravaccinia  NOS 
Varicella  complication  NEC 
Varicella  complication  NOS 
Varicella  uncomplicated 
Measles  w/complication  NEC 
Measles  w/complication  NOS 
Measles  uncomplicated 
Rubella  w/complication  NEC 
Rubella  w/complication  NOS 
Rubella  uncomplicated 
Erythema  infectiosum 
Exanthemata,  viral  NEC 
Exanthemata,  viral  NOS 
Hand,  foot  and  mouth  disease 
Fever,  spotted 
Rickettsialpox 
Erythema,  toxic 
Erythema  multiforme 
Erythema  nodosum 
Erythematous  conditions  NEC 
Erythematous  condition  NOS 


:Resp 

Dx 

Pneumonia,  salmonella 
Plague,  primary  pneumonic 
Plague,  secondary  pneumonic 
Plague,  pneumonic  NOS 
Tularemia,  pulmonary 
Anthrax,  pulmonary 
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031.0  Disease,  pulmonary  d/t  mycobacteria 

031.8  Disease,  mycobacterial  NEC 

031.9  Disease,  mycobacterial  NOS 

032.0  Diphtheria,  faucial 

032.1  Diphtheria,  nasopharyngeal 

032.2  Diphtheria,  anterior  nasal 

032.3  Diphtheria,  laryngeal 

032.89  Diphtheria  NEC 

032.9  Diphtheria  NOS 

033.0  Whoopcough,  Bordetella  pertussis 

033.1  Whoopcough  Bordetella  parapertussis 

033.8  Whooping  cough  NEC 

033.9  Whooping  cough  NOS 

034.0  Sore  throat,  streptococcal 

052.1  Varicella  pneumonitis 

055.1  Pneumonia,  postmeasles 

055.2  Otitis  media,  postmeasles 

073.0  Ornithosis  w/pneumonia 

079.0  Infection,  adenovirus 

079.1  Infection,  ECHO  virus 

079.2  Infection,  Coxsackie  virus 

079.3  Infection,  rhinovirus 

079.6  Infct,  respiratory  syncytial  virus 

079.82  SARS  ASSOC  CORONAVIRUS 

381.00  OM,  acute  nonsuppurative  NOS 

381.01  OM,  acute  serous 

381.03  OM,  acute  sanguinous 

381.04  OM,  acute  allergic  serous 

381.4  OM,  chronic  nonsuppurative  NOS 

381.50  Salpingitis,  Eustachian  NOS 

381.51  Salpingitis,  acute  Eustachian 

382.00  OM,  acute  suppurative  NOS 

382.01  OM,  acute  suppurative  w/drum  rup 

382.02  OM,  acute  suppurative  in  disease  CE 

382.4  OM,  suppurative  NOS 

382.9  Otitis  media  NOS 

460  Nasopharyngitis,  acute 

461.0  Sinusitis,  acute  maxillary 

461.1  Sinusitis,  acute  frontal 

461.2  Sinusitis,  acute  ethmoidal 

461.3  Sinusitis,  acute  sphenoidal 

461.8  Sinusitis,  acute  NEC 

461.9  Sinusitis,  acute  NOS 

462  Pharyngitis,  acute 

463  Tonsillitis,  acute 

464.00  Laryngitis,  Acute,  w/o  obstruction 

464.01  Laryngitis,  Acute. W/  obstruction 

464.10  Tracheitis,  acute,  w/o  obstruction 

464.11  Tracheitis,  acute  w/obstruction 

464.20  Laryngotracheitis ,  acute  w/o  obst 

464.21  Laryngotracheitis,  acute  w/obst 

464.30  Epiglottitis,  acute  w/o  obst 

464.31  Epiglottitis,  acute  w/obstruction 

464.4  Croup 

464.50  Supraglottis,  uns.  w/out  obstr. 

464.51  Supraglottis,  uns.  w/  obstr. 

465.0  Laryngopharyngitis ,  acute 
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465.8  Infct  up  rsprt  mlt  sites,  acute  NEC 

465.9  Infct  up  rsprt  mlt  sites,  acute  NOS 

466.0  Bronchitis,  acute 

466.11  Bronchiolitis,  acute,  d/t  RSV 

466.19  Bronchio  acute  d/t  oth  infct  orgnsm 

478.9  Disease,  upper  respiratory  NEC/NOS 

480.0  Pneumonia,  adenovirus 

480.1  Pneumonia  d/t  rsprt  syncytial  virus 

480.2  Pneumonia  d/t  parainfluenza  virus 

480.3  PNEUMONIA  DUE  TO  SARS 

480.8  Pneumonia  d/t  virus  NEC 

480.9  Pneumonia  d/t  virus  NOS 

481  Pneumonia  d/t  pneumococcal  virus 

482.0  Pneumonia  d/t  Klebsiella  pneumoniae 

482.1  Pneumonia  d/t  Pseudomonas 

482.2  Pneumonia  d/t  Hemophilus  influenzae 

482.30  Pneumonia  d/t  Streptococcus  NOS 

482.31  Pneumonia  d/t  Streptococcus  Group  A 

482.32  Pneumonia  d/t  Streptococcus  Group  B 

482.39  Pneumonia  d/t  Streptococcus  NEC 

482.40  Pneunonia  d/t  Staphylococcus  NOS 

482.41  Pneumonia  d/t  Staphylococcus  aureus 

482.49  Pneumonia  d/t  Staphylococcus  NEC 

482.81  Pneumonia  d/t  anaerobes 

482.82  Pneumonia  d/t  Escherichia  coli 

482.83  Pneumonia  d/t  gram-negative  NEC 

482.84  Pneumonia  d/t  Legionnaires'  disease 

482.89  Pneumonia,  bacterial  NEC 

482.9  Pneumonia,  bacterial  NOS 

483.0  Pneumonia  d/t  Mycoplasma  pneumoniae 

483.1  Pneumonia  d/t  Chlamydia 

483.8  Pneumonia  d/t  organism  NEC 

484.1  Pneumonia  in  cytomegalic  incls  dis 

484.3  Pneumonia  in  whooping  cough 

484.5  Pneumonia  in  anthrax 

484.6  Pneumonia  in  aspergillosis 

484.7  Pneumonia  in  systemic  mycoses 

484.8  Pneumonia  in  oth  infct  disease  CE 

485  Bronchopneumonia,  organism  NOS 

486  Pneumonia,  organism  NOS 

487.0  Influenza  w/pneumonia 

487.1  Influenza  w/rsprt  mnfst  NEC 

487.8  Influenza  w/manifestation  NEC 

490  Bronchitis  NOS 

494.1  BRONCHIECTASIS  W  AC  EXAC 

511.0  Pleurisy  w/o  effusion  or  TB 

511.1  Pleurisy,  w/bctrl  effusion,  not  TB 

511.8  Pleurisy,  effusion  NEC,  not  TB 

511.9  Effusion,  pleural  NOS 

513.0  Abscess,  lung 

513.1  Abscess,  mediastinum 

514  Congestion/hypostasis,  pulmonary 

517.3  ACUTE  CHEST  SYNDROME 

518.0  Collapse,  pulmonary 

518.4  Edema,  acute  lung  NOS 

518.81  Failure,  acute  respiratory 

518.82  Insufficiency,  pulmonary  NEC 
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518 . 84 

519.2 

519.3 
769 
782.5 

784.1 
786.00 
786.05 
786.06 
786.07 
786.09 

786.1 

786.2 

786.3 
786.52 
786.7 
786.9 


Syndrome 

icd9 

040 . 82 

458 . 9 
780.01 
785.50 
785.52 
785.59 

798.1 

798.2 

798 . 9 
799.1 


Respiratory  failure, acute  &  chronic 

Mediastinitis 

Disease,  mediastinum  NEC 

Syndrome,  respiratory  distress 

Cyanosis 

Pain,  throat 

Abnormality,  respiratory  NOS 
Shortness  of  breath 
Tachypnea 
Wheezing 

Abnormality,  respiratory  NEC 

Stridor 

Cough 

Hemoptysis 

Painful  respiration 

Abnormal  chest  sounds 

Symp  inv  respiratory  syst/chest  NEC 


Shk-Coma 

Dx 

TOXIC  SHOCK  SYNDROME 
Hypotension  NOS 
Coma 

Shock  NOS 

SEPTIC  SHOCK 

Shock  w/o  trauma  NEC 

Death,  instantaneous 

Death,  less  than  24  hrs  onset  symp 

Death,  unattended 

Arrest,  respiratory 
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Appendix  4:  DoD  synonyms  for  test  names 


New  Test  Name 

New  Print  Name 

Synonym 

ACANTHAMOEBA  CULTURE 

ACANTHAM  CULT 

CULTURE  ACANTHAMOEBA 

ACANTHAMOEBA/NAEGLERIA 

CULTURE 

CULT 

ACANTH/NAEG 

ACANTHAMEOBA/NAEGLERIA 

ACTIN  THERMOPHILIC 

COLONY  CT 

ACTIN  THERMOPHI 

AEROBIC  CULTURE 

AEROBIC  CULT 

AEROBIC  BLOOD  CULT; 

AEROBIC  BC;  A  BC;  AEROBIC 
BODY  FLUID  CULT;  AEROBIC  BF 
CULT;  A  BF  CULTURE; 

CULTURE  AEROBIC 

ANAEROBIC  CULTURE 

ANAER  CULT 

ANAEROBIC  BLOOD  CULT; 
ANAEROBIC  BC;  ANA  BC; 
ANAEROBIC  BODY  FLUID  CULT; 
ANAEROBIC  BF  CULT;  ANA  BF 
CULTURE;  CULTURE 

ANAEROBIC 

B  ANTHRACIS  BETAPHAGE 

ANTHRAX  PHAGE 

BACILLUS  ANTHRACIS; 

ANTHRAX  PHAGE 

B  ANTHRACIS  CULTURE 

ANTHRAX  CULT 

BACILLUS  ANTHRACIS; 

ANTHRAX  CULTURE;  CULTURE, 
ANTHRAX 

B  ANTHRACIS  DFA 

ANTHRAX  DFA 

BACILLUS  ANTHRACIS; 

ANTHRAX  DFA 

B  ANTHRACIS  ID 

B  ANTHRACIS  ID 

ANTHRAX 

B  ANTHRACIS  PCR 

ANTHRAX  PCR 

BACILLUS  ANTHRACIS; 

ANTHRAX  PCR 

B  BURGDORFERI 

B  BURGDORFERI 

LYMES  DISEASE 

B  DERMATITIDIS  EXO-AG  ID 

B  DERM  EXO-AG 

B  HENSELAE  CULTURE 

B  HENSELAE  CULT 

BARTONELLA  HENSELAE 
CULTURE 

B  HENSELAE  H-1 

B  HENSELAE  H-1 

B  HENSELAE  HI ; BARTONELLA 
HENSELAE  H-1 

B  PERTUSSIS  CULTURE 

PERTUSSIS  CULT 

BORDETELLA  PERTUSSIS 
CULTURE 

B  QUINTANA  OK 

B  QUINTANA  OK 

B  QUINTANA  OKLAHOMA 

BACTERIA 

BACTERIA 

CULTURE,  BACTERIA 

BACTERIA  ID  GAS  CHROM 

BACTERIA  ID  GC 

BACTERIA  ID  PFGE 

BACTERIA  ID  PFG 

BARTONELLA  CULTURE 

BARTON  CULT 

CULTURE  BARTONELLA 

BETA  STREP  ALLERGIC 

STREP  ALLERGIC 

STREPTOCOCCUS  B  PCN 
ALLERGIC;  BETA  STREP-PCN 
ALLERGIC 

BLASTOMYCES  ID 

BLASTO  ID 

BLASTOMYCOSIS 

BLOOD  CULTURE 

BLD  CULT 

BC;  PED  BC;  PEDIATRIC  BLOOD 
CULTURE;  CULTURE  BLOOD 

BLOOD  CULTURE 

BC  ANAEROBIC;  ANAEROBIC 

ANAEROBIC 

BLD  CULT  ANA 

BLOOD  CULTURE 

BODY  FLUID  CULT 

BF  CULT 

BF  CULTURE;  CULTURE  BODY 
FLUID 
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BORDETELLA  CULTURE 

BORDETELLA  CULT 

WHOOPING  COUGH 

BORDETELLA  PERTUSSIS 

B  PERTUSSIS 

WHOOPING  COUGH; 
BORDETELLA  PERTUSSIS 
CULTURE 

BORDETELLA  SP  CULTURE 

BORDTELAS  CULT 

BORDETELLA  SP  CULTURE 

BORDETELLA  SP  ID 

BORDETEL  SP  ID 

BORRELIA  SP  CULTURE 

BORRELIA  S  CULT 

BORRELIA  SP  CULTURE 

BORRELIA  SP  ID 

BORRELIA  SP  ID 

BRUCELLA  CULTURE 

BRUCELLA  CULT 

BRUCELLOSIS 

BURKHOLDERIA  CULTURE 

BURKHOLD  CULT 

C  DIFFICILE 

C  DIFFICILE 

CLOSTRIDIUM  DIFFICILE 

C  DIPHTHERIAE  CULTURE 

C  DIPHTHERCULT 

CLOSTRIDIUM  DIPHTHERIA; 

CORYNEBACTERIUM 

DIPHTHERIAE 

C  TRACHOMATIS 

C  TRACHOMATIS 

CHLAMYDIA  TRACHOMATIS 

CALYMMATOBACTERIUM 

GRANULOMTIS 

C  GRANULOMTIS 

CAMPYLOBACTER  ID 

CAMPY  ID 

CAMPYLOBACTER  SP  ID 

CAMPY  SP  ID 

CATALASE  TEST 

CATALASE 

CERVICAL  MUCUS 

CERVICAL  MUCUS 

CHLAMYDIA  CULTURE 

CHLAMYDIA  CULT 

CHLAMYDIA  SP  IDENTIFIED 

CHLAMYDIA  SP  ID 

CHLAMYDIA  SP  ID 

CHLORACETATE  ESTERASE 
STAIN 

CE  STAIN 

CLO  TEST 

CLO  TEST 

CLOTEST 

CLOTEST 

H  PYLORI;  HELICOBACTER 
PYLORI 

COCCI  IMMITIS  EXOAG  ID 

C  IMMITIS  EXOAG 

COCCIDIOIDES  ID 

COCCIDIA  ID 

COCCIDIOIDES  IDCF  ID 

COCCIDIO  IDCF 

COCCIDIOIDES  IDTP  ID 

COCCIDIO  IDTP 

COLONY  COUNT 

COLONY  COUNT 

CRYPTOSPORIDIUM 

CRYPTOSPORIDIUM 

CSF  CULT 

CSF  CULTURE 

CULTURE  CSF;  SPINAL  FLUID 
CULTURE 

CYANOBACTERIUM  ID 

CYANOBTERIUM  ID 

CYCLOSPORA  CYAETINESUS 

C  CYAETINESUS 

CYCLOSPORA  ID 

CYCLOSPORA  ID 

DIPHTHERIA  SP  ID 

DIPHTHERIA  SP 

E  COLI  ENTERO  ID 

E  COLI  ENTER  ID 

ESCHERICHIA  COLI 
ENTEROHEMORRHAGIC  ID 

E  COLI  0157:H7 

E  COLI  0157:H7 

ESCHERICHIA  COLI  0157:H7 

E  COLI  0157:H7  ID 

E  COLI  0157:H7 

ESCHERICHIA  COLI  0157:H7  ID 

E  TEST 

E  TEST 

E  TEST  SUSCEPTIBILITY; 
SUSCEPTIBILITY 

EAR  CULTURE 

EAR  CULT 

CULTURE  EAR 

ENTEROVIRUS  ID 

ENTEROVIRUS  ID 

ENVIRON  CULT  BT 

ENVIRON  CULT  BT 

EYE  CULTURE 

EYE  CULT 

CULTURE  EYE 
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F  TULARENSIS  CULTURE 

TULARENSIS  CULT 

TULAREMIA 

GC  CULT 

GC  CULT 

N  GONORRHOEAE;  GC; 
NEISSERIA  GONORRHOEAE; 
GONORRHEA  CULTURE 

GC  SMEAR 

GC  SMEAR 

N  GONORRHOEAE;  GC; 
NEISSERIA  GONORRHOEAE 

GENITAL  CULTURE 

GENITAL  CULTURE 

CULTURE  GENTIAL 

GRAM  STAIN 

GRAM  STAIN 

H  PYLORI  CULTURE 

H  PYLORI  CULT 

H  PYLORI  CULTURE; 
CAMPYLOBACTER  PYLORI 
CULTURE;  HELICOBACTER 
PYLORI  CULTURE 

HAEMOPHILUS  B  CULTURE 

HAEMOPH  B  CULT 

HAEMOPHILUS  B  CULTURE 

HAEMOPHILUS  DUCREYI 
CULTURE 

H  DUCREYI  CULT 

H  DUCREYI  CULTURE 

HAEMOPHILUS  SP 

IDENTIFIED 

HAEMOPHIL  SP  ID 

HAEMOPHILUS  SP  ID 

HISTOPLASMA  CULTURE 

HISTOPLASM  CULT 

LEGIONELLA  CULTURE 

LEGIONELLA  CUL 

LEGIONELLA 

LEGIONELLA  SP 

LEGIONELLA  SP 

LEGIONELLA  SP  IDENTIFIED 

LEGION  SP  ID 

LEGIONELLA  SP  ID 

LEPTOSPIRA  SP  ID 

LEPTOSPIR  SP  ID 

LISTERIA  SP  ID 

LISTERIA  SP  ID 

M  HOMINIS 

M  HOMINIS 

MYCOPLASMA  HOMINIS 

M  PNEUMONIAE  CULTURE 

M  PNEUMO  CULT 

MYCOPLASMA  PNEUMONIAE 
CULTURE 

METH  RESISTANT  S  AUREUS 

MRSA 

MRSA 

MICROORGANISM 

IDENTIFIED 

MICROORG  ID 

MICROORGANISM  ID 

MYCOPLASMA  SP  GENITAL  ID 

MYCOPLAS  GENITA 

MYCOPLASMA  SP  GENITAL  ID 

MYCOPLASMA  SP  IDENTIFIED 

MYCOPLASM  SP  ID 

MYCOPLASMA  SP  ID 

MYCOPLASMA  SP  RESP  ID 

MYCOPLAS  RESP 

MYCOPLASMA  SP  RESP  ID 

MYCOPLASMA/UREAPLASMA 

CUL 

MYCOPLASMA  CULT 

MYCOPLASMA+U  REAPLASMA 
SP 

MYCOPLASMA+U REA 

N  GONORRHOEAE 

N  GONORRHOEAE 

N  GONORRHOEAE;  GC; 
NEISSERIA  GONORRHOEAE 

NASAL  CULTURE 

NASAL  CULT 

CULTURE  NASAL 

NORMAL  SALINE 

SALINE 

SALINE 

ORGANISM  COUNT 

ORGANISM  CNT 

ORGANISM  IDENTIFICATION 

ORGANISM  ID 

RECTAL  CULTURE 

RECTAL  CULT 

CULTURE  RECTAL 

RESPIRATORY  CULTURE 

RESP  CULT 

CULTURE  RESPIRATORY, 
NASOPHARYNGEAL  CULTURE 

SALMONELLA  SEROGROUP 

SALMONELLA  GRP 

SEROTYPING  H  INFLUENZAE 

TYPING  H  FLU 

TYPE/GROUP  H  INFLUENZAE;  H 
INFLUENZAE  SEROTYPING 

SEROTYPING  N  MENINGITIS 

TYPING  N  MEN 

TYPE/GROUP  N  MENINGITIS;  N 
MENINGITIS  SEROTYPING 

SEROTYPING  OTHER 
ORGANISM 

TYPING  OTHER 

TYPE/GROUP  OTHER 

ORGANISM 
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SEROTYPING  STAPH 

AUREUS 

TYPING  SA 

TYPE/GROUP  STAPH  AUREUS; 
STAPH  AUREUS  SEROTYPING 

SEROTYPING 

STREPTOCOCCUS 

TYPING  STREP 

TYPE/GROUP 

STREPTOCOCCUS; 

STREPTOCOCCUS 

SEROTYPING 

SEROTYPING  VIBRIO 

TYPING  VIBRIO 

TYPE/GROUP  VIBRIO;  VIBRIO 
SEROTYPING 

SHIGELLA  SEROTYPE 

SHIGELLA  SERO 

RAST;  ALLERGEN 

SPORE  STRIP 

SPORE  STRIP 

ATTEST 

SPUTUM  CULTURE 

SPUTUM  CULT 

CULTURE  SPUTUM 

STERILITY  TEST 

STERILITY  TEST 

STOOL  CULTURE 

STOOL  CULT 

CULTURE  STOOL 

STREP  AGALACTIAE  ID 

S  AGALACTIAE  ID 

STREPTOCOCCUS  B- 
HEMOLYTIC 

STREP  B-HEMOLY 

BETA  HEMOLYTIC 
STREPTOCOCCUS 

SUSCEPTIBILITY  AEROBIC 

SUSC  AER 

AEROBIC  SUSCEPTIBILITY 

SUSCEPTIBILITY  ANAEROBIC 

SUSC  ANA 

ANAEROBIC  SUSCEPTIBILITY 

TARCROLINUS 

TARCROLINUS 

THROAT  CULTURE 

THR  CULT 

TC;  CULTURE  THROAT 

TISSUE  CULTURE 

TISSUE  CULT 

CULTURE  TISSUE 

TOXOPLASMA  SP 

TOXOPLASMA  SP 

TOXOPLASMA  SP  ID 

TOXOPLASM  SP  ID 

UREAPLASMA  UREAL YTICUM 
CULT 

U  UREALYTICUM 

URINE  CULTURE 

UA  CULT 

UC;  CULTURE  URINE 

VANCO  INTERMED  S  AUREUS 

VISA 

VISA 

VANCO  RESIST 
ENTEROCOCCUS 

VRE  ID 

VANCOMYCIN  RESISTANT 
ENTEROCOCCUS;  VRE  SCREEN 

VANCO  RESIST  S  AUREUS 

VRSA 

VRSA 

VIBRIO  SP  ID 

VIBRIO  SP  ID 

WATER  CULTURE 

H20  CULT 

CULTURE  WATER 

WOUND  CULTURE 

WND  CULT 

WC;  SUPERFICIAL  WOUND; 
SUPERFICIAL  WND  CULT; 
ABSCESS  CULTURE 

WOUND  CULTURE  DEEP 

DP  WND  CULT 

DEEP  WND  CULTURE;  DEEP 

WND  CULT;  DEEP  WC; 

CULTURE  WOUND 

YERSINIA  CULTURE 

YERSINIA  CULT 
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Appendix  5:  Standardized  Test  Names 


Laboratory  test 

order  classification 

std  test  ord 

tstorder 

COUNT 

PERCENT 

ACINETOBACTER 

ACINETO  SCREEN 

8 

0.000 

AER  BLD  CULT 

AERO  BC 

255 

0.014 

AERO  BLD  CULT 

660 

0.037 

AEROBIC  BC 

607 

0.034 

AEROBIC  BLD 

469 

0.026 

AEROBIC  BLD  CUL 

1353 

0.076 

AEROBIC  CULT, BLOOD 

224 

0.013 

AEROBIC  CULTURE, BLOOD 

228 

0.013 

BC  (AER) 

3248 

0.182 

BC  AER 

92 

0.005 

BLD  CULT,  AEROB 

695 

0.039 

BLD, AEROBIC 

3520 

0.197 

BLOOD  CX, AERO 

833 

0.047 

C  AEROBIC  BLD 

667 

0.037 

AER  BLD  CULT 

12851 

0.720 

AEROBIC  CULT 

AER  CULT 

200 

0.011 

AEROBIC  CULT 

10095 

0.566 

AEROBIC  CULTURE 

732 

0.041 

C  AEROBIC 

3077 

0 . 172 

AEROBIC  CULT 

14104 

0.790 

AFB 

ACID  FAST  CULT 

2 

0.000 

AFB 

2 

0.000 

AFB  BC 

1 

0.000 

AFB  BLD  CX 

5 

0.000 

AFB  CONC 

22 

0.001 

AFB  CUL  BLOOD 

9 

0.001 

AFB  CULT 

7997 

0.448 

AFB  CULTURE 

522 

0.029 

AFB  ID 

1 

0.000 

AFB  PAN-APATH 

50 

0.003 

AFB  PANEL 

1068 

0.060 

AFB  SMEAR 

15 

0.001 

AFB  SMEAR/ CULTU 

54 

0.003 

AFB  SMR  AURAMIN 

82 

0.005 

AFB  SMR  KINYOUN 

1 

0.000 

AFB  S MR/CULT 

450 

0.025 

AFB  SPUTUM 

5 

0.000 

AFB  ST/CULT 

5 

0.000 

AFB  STAIN 

111 

0.006 

AFB  STAIN/CUL 

11 

0.001 

AFB-BNH 

5 

0.000 

BLD  CULT-CAFB 

19 

0.001 

C  AFB 

182 

0.010 

MYCO  (WBG) 

1 

0.000 

MYCOBTER  ID 

108 

0.006 

TB  CULT 

6 

0.000 

TB  ISO  BLD  CULT 

10 

0.001 

WBC-CAFB 

21 

0.001 
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Laboratory  test 

std  test  ord 

order  classification 

tstorder 

COUNT 

PERCENT 

AFB 

10765 

0.603 

ANAER  BLD  CULT 

ANA  BC 

3 

0.000 

ANA  BLD 

7 

0.000 

ANAER  CULT, BLOOD 

181 

0.010 

ANAEROBBLD  CULT 

651 

0.036 

ANAEROBIC  BC 

393 

0 . 022 

ANAEROBIC  BLD 

465 

0.026 

ANAEROBIC  CULT, BLOOD 

6 

0.000 

ANAEROBIC  CULTU, BLOOD 

1 

0.000 

BC  ANA 

1079 

0.060 

BLD  CULT , ANAERO 

600 

0.034 

BLD, ANAEROBIC 

995 

0.056 

BLOOD  CX  ANA 

787 

0.044 

ANAER  BLD  CULT 

5168 

0.290 

ANAER  CULT 

ANA  CULT 

62 

0.003 

ANAER  CULT 

3234 

0.181 

ANAER  CX 

1 

0.000 

ANAEROB 

6 

0.000 

ANAEROBE  CULTUR 

51 

0.003 

ANAEROBE-APATH 

91 

0.005 

ANAEROBIC  C&S 

113 

0.006 

ANAEROBIC  CULT 

223 

0.012 

ANAEROBIC  CX 

11 

0.001 

ANAEROBIC  PANEL 

1438 

0.081 

C  ANAEROBIC 

515 

0.029 

ANAER  CULT 

5745 

0 . 322 

BF  CULT 

AER  CULT , BF 

12 

0.001 

AEROBIC  CULT , BF 

759 

0.043 

ANA  CULT , BF 

12 

0.001 

ANAER  CULT , BF 

70 

0.004 

ANAEROBIC  PANEL, BF 

3 

0.000 

BDY  FLD 

4 

0.000 

BDY  FLD  CULTURE 

129 

0.007 

BF  CULT 

1175 

0.066 

BF  CX 

14 

0.001 

BODY  FL  CULTURE 

31 

0.002 

BODY  FLUID 

174 

0.010 

BODY  FLUID  CUL 

8 

0.000 

BODY  FLUID  CULT 

420 

0.024 

C  BODY  FLUID 

405 

0.023 

C&S  BDY  FLD  OTH 

1138 

0.064 

CUL  BODY  FLUID 

48 

0.003 

CULTURE, BF 

79 

0.004 

FL  C&S 

26 

0.001 

FLD  CULT 

39 

0.002 

FLUID 

23 

0.001 

FLUID  C&S 

148 

0.008 
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Laboratory  test  order  classification 


std  test  ord 

tstorder 

COUNT 

PERCENT 

BF  CULT 

FLUID  CULT 

2665 

0.149 

FLUID  CULTURE 

225 

0.013 

MISC  CULTURE, BF 

9 

0.001 

SBF  CULTURE 

4 

0.000 

STERILE  SITE  CX 

778 

0.044 

STERILE  SITEC&S 

1 

0.000 

SYN  CUL 

2 

0.000 

SYNOVIAL  CX , BEN 

3 

0.000 

BF  CULT 

8404 

0.471 

BLD  CULT 

ADULT  BLD  CULT 

11 

0.001 

BC  A/ AN 

385 

0 . 022 

BC  AER  &  ANAER 

67 

0.004 

BC  PED  AER 

176 

0.010 

BC  SET 

35 

0.002 

BC  W/RES 

212 

0.012 

BLD  CLT  ANA/AER 

476 

0.027 

BLD  CULT 

109107 

6.113 

BLD  CULT  PANEL 

766 

0.043 

BLD  CULT  SET 

191 

0.011 

BLD  CULT, ADULT 

349 

0.020 

BLD  CX 

804 

0.045 

BLOOD  CUL 

1834 

0.103 

BLOOD  CULT  PEDI 

225 

0.013 

BLOOD  CULTURE 

10399 

0.583 

BLOOD  CULTURE  P 

718 

0.040 

BLOOD  CX 

1647 

0.092 

BLOOD  CX, PEDS 

1695 

0.095 

BLOOD,  R/O  SBE 

21 

0.001 

C  BLOOD 

2935 

0.164 

ISO  BLOOD  CULT 

8 

0.000 

LEPTO  CULTURE, BLOOD 

11 

0.001 

MISC.  C, BLOOD 

1 

0.000 

OTHER  CULTURE, BLOOD 

6 

0.000 

PED  BLOOD  CULT 

385 

0 . 022 

PEDS  BC 

315 

0.018 

SBE  SUBCULTURE 

409 

0.023 

TRAN  RX  CUL, BLOOD 

4 

0.000 

BLD  CULT 

133192 

7.462 

BLD  PARA 

BLOOD  PARASITES 

248 

0.014 

MALARIA 

74 

0.004 

MALARIA  SMEAR 

156 

0.009 

MICROFILARIA  SP 

1 

0.000 

BLD  PARA 

479 

0.027 

BORDETELLA  CULT 

B  PERTUSSIS 

23 

0.001 

B .  PERT 

1 

0.000 

BORDETELLA  CULT 

6 

0.000 

BORDETELLA  DFA 

31 

0.002 
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Laboratory  test  order  classification 


std  test  ord 

tstorder 

COUNT 

PERCENT 

BORDETELLA  CULT 

BORDETELLA  SMEA 

2 

0.000 

BORDTELA  S  CULT 

37 

0.002 

PERTUS.  CULT 

9 

0.001 

PERTUSS, FA 

3 

0.000 

PERTUSSIS 

83 

0.005 

PERTUSSIS  CULT 

34 

0.002 

BORDETELLA  CULT 

229 

0.013 

BRUCELLA  CULT 

BRUCELLA  CULT 

11 

0.001 

C&S  BRUCE LL/TUL 

2 

0.000 

BRUCELLA  CULT 

13 

0.001 

C  DIFFICILE 

C  DIFF  A+B 

6 

0.000 

C  DIFFICILE 

119 

0.007 

C-DIFFICILE 

54 

0.003 

C.  DIFFICILE 

108 

0.006 

C. DIFFICILE  TOX 

48 

0.003 

C  DIFFICILE 

335 

0.019 

CATH  CULT 

C  CATH 

19 

0.001 

C&S  IV  CATH  TIP 

20 

0.001 

CATH  TIP 

5 

0.000 

CATH  TIP  C&S 

3 

0.000 

CATH  TIP  CULT 

71 

0.004 

CATHETER  CULT 

2 

0.000 

CATHETER  TIP 

3 

0.000 

CATH  CULT 

123 

0.007 

CERVICAL  CULT 

CERV/STREP 

2 

0.000 

CERV/VAG 

231 

0.013 

CERVICAL  CULTUR 

54 

0.003 

CERVICAL  CULT 

287 

0.016 

CHLAMYDIA  CULT 

CHLAMYDIA  CULT 

252 

0.014 

CHLAMYDIA  PROBE 

353 

0.020 

CHLAMYDIA  CULT 

605 

0.034 

CMV 

CMV  CULTURE 

97 

0.005 

CMV  VR  C/S 

4 

0.000 

CMV 

101 

0.006 

CSF  CULTURE 

ACANTHAMOEBA  CU 

30 

0.002 

ANAEROBIC  PANEL, CSF 

3 

0.000 

C  CSF 

753 

0 . 042 

CSF 

46 

0.003 

CSF  CUL/GRAM  ST 

63 

0.004 
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Laboratory  test  order  classification 


std  test  ord 

tstorder 

COUNT 

PERCENT 

CSF  CULTURE 

CSF  CULT 

50 

0.003 

CSF  CULT-BNH 

133 

0.007 

CSF  CULTURE 

10196 

0.571 

CSF  PANEL 

2135 

0.120 

CSF  SMEAR 

5 

0.000 

CSF  VR  PNL 

38 

0.002 

CSF  CULTURE 

13452 

0.754 

E  COLI  0157 : H7 

E  COLI  0157 :H7 

2 

0.000 

E  COLI  0157 :H7 

3 

0.000 

E , COLI  0157 :H7 

67 

0.004 

E.C.  0157  CULT 

12 

0.001 

E . COLI  0157 :H7 

61 

0.003 

E  COLI  0157 :H7 

145 

0.008 

EAR  CULT 

C  EAR 

113 

0.006 

C&S  EAR 

167 

0.009 

EAR  C&S 

261 

0.015 

EAR  CUL 

97 

0.005 

EAR  CULT 

817 

0.046 

EAR  CULTURE 

38 

0.002 

EAR  CULTURE  PAN 

40 

0.002 

EAR  CX 

33 

0.002 

EAR  CULT 

1566 

0.088 

ENVIRON  CULT  BT 

ENVIRO  CULT 

22 

0.001 

ENVIRON  CULT  BT 

421 

0.024 

ENVIRON  CULTURE 

2 

0.000 

ENVIRONMENT  CUL 

3 

0.000 

ENVIRON  CULT  BT 

448 

0.025 

EYE  CULT 

C  EYE 

288 

0.016 

C&S  EYE 

460 

0.026 

EYE  C&S 

255 

0.014 

EYE  CU 

117 

0.007 

EYE  CULT 

1222 

0.068 

EYE  CULTURE 

225 

0.013 

EYE  CULTURE  PAN 

21 

0.001 

EYE  CX 

74 

0.004 

EYE  CULT 

2662 

0.149 

FECAL  RS 

FEC  REDUCING  SU 

15 

0.001 

FECAL  RED  SUBST 

5 

0.000 

RED  SUB 

1 

0.000 

RED  SUBST 

2 

0.000 

REDUCE  STL 

2 

0.000 

REDUCING 

17 

0.001 

REDUCING  SUBST 

17 

0.001 
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Laboratory  test  order  classification 


std  test  ord 

tstorder 

COUNT 

PERCENT 

FECAL  RS 

REDUCING  SUBT 

12 

0.001 

STOOL  RED  SUBST 

3 

0.000 

FECAL  RS 

74 

0.004 

FECAL  WBC 

FEC  WBC 

923 

0.052 

FECAL  LEUKOCYTE 

1609 

0.090 

FECAL  WBC 

1450 

0.081 

FECAL  WBCS 

17 

0.001 

FECAL WBC 

4 

0.000 

STOOL  WBC'S 

206 

0.012 

STOOLWBC 

232 

0.013 

WBC  '  S 

45 

0.003 

FECAL  WBC 

4486 

0.251 

FOB 

FOB 

2157 

0.121 

OC  BLD 

639 

0.036 

OCC  BL 

4579 

0.257 

OCC  BLD 

155 

0.009 

OCC  BLD  X3 

315 

0.018 

OCC  BLDX3 

2848 

0.160 

OCCBDX3 

1242 

0.070 

OCCBLDX3 

184 

0.010 

OCCULT 

2329 

0.130 

OCCULT  BLOOD 

28138 

1.576 

OCCULT  BLOOD  X3 

204 

0.011 

FOB 

42790 

2 . 397 

FUNGUS,  CSF 

C  FUNGAL, CSF 

1 

0.000 

CRYPTOCOCCUS  AG, CSF 

63 

0.004 

FUNGAL  CULT, CSF 

31 

0.002 

FUNGAL  CULTURE, CSF 

73 

0.004 

FUNGAL  MI SC, CSF 

54 

0.003 

FUNGAL  SMEAR, CSF 

1 

0.000 

FUNGAL, CSF 

53 

0.003 

FUNGI  YEASTLIKE, CSF 

1 

0.000 

FUNGUS  MICRO, CSF 

1 

0.000 

INDIA  IN, CSF 

1 

0.000 

INDIA  INK, CSF 

41 

0.002 

INDIA  INK, CSF, CSF 

1 

0.000 

INDIA  INK-CSF, CSF 

9 

0.001 

KOH  PREP, CSF 

1 

0.000 

MYCOL  C&SM, CSF 

13 

0.001 

MYCOLOGY  CUL, CSF 

3 

0.000 

MYCOLOGY  CULT, CSF 

30 

0.002 

MYCOLOGY, CSF 

4 

0.000 

FUNGUS,  CSF 

381 

0.021 

FUNGUS,  GENITAL 

C  FUNGAL, GEN 

11 

0.001 

FUNGAL  CULT, GEN 

80 

0.004 
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Laboratory  test  order  classification 


std  test  ord 

tstorder 

COUNT 

PERCENT 

FUNGUS,  GENITAL 

FUNGAL  CULT , GEN , GEN 

5 

0.000 

FUNGAL  CULTURE, GEN 

120 

0.007 

FUNGAL  CULTURE, GEN, GEN 

12 

0.001 

FUNGAL  MI SC, GEN 

75 

0.004 

FUNGAL  MI SC , GEN , GEN 

6 

0.000 

FUNGAL  SMEAR, GEN 

125 

0.007 

FUNGAL, GEN 

46 

0.003 

FUNGAL, GEN, GEN 

3 

0.000 

FUNGAL- BNH, GEN 

3 

0.000 

FUNGI  YEASTLIKE, GEN 

24 

0.001 

FUNGI  YEASTLIKE, GEN, GEN 

2 

0.000 

FUNGUS  ID, GEN 

3 

0.000 

KOH  /  WET  PREP, GEN 

108 

0.006 

KOH  /  WET  PREP, GEN, GEN 

18 

0.001 

KOH  PREP, GEN 

9526 

0.534 

KOH  PREP, GEN, GEN 

744 

0 . 042 

KOH, GEN 

22941 

1.285 

KOH, GEN, GEN 

730 

0.041 

KOH/NS, GEN 

315 

0.018 

KOH/NS, GEN, GEN 

13 

0.001 

KOH/WET  PREP, GEN 

772 

0.043 

KOH/WET  PREP, GEN, GEN 

281 

0.016 

MYCOL  C&SM, GEN 

4 

0.000 

MYCOL  C&SM, GEN, GEN 

2 

0.000 

MYCOLOGY  CULT, GEN 

80 

0.004 

MYCOLOGY  CULT , GEN , GEN 

4 

0.000 

MYCOLOGY  SMEAR, GEN 

4 

0.000 

MYCOLOGY, GEN 

15 

0.001 

NS  PREP, GEN 

4570 

0.256 

FUNGUS,  GENITAL 

40642 

2.277 

FUNGUS,  OTHER 

B/M  CUL  FUNGAL 

1 

0.000 

BLD  CUL  FUNGAL 

7 

0.000 

C  FUNGAL 

164 

0.009 

DFA  CRYPTO 

34 

0.002 

FUNG  ID 

19 

0.001 

FUNGAL 

3377 

0.189 

FUNGAL  BC 

2 

0.000 

FUNGAL  BLD 

5 

0.000 

FUNGAL  BLD  CULT 

25 

0.001 

FUNGAL  CUL 

5791 

0.324 

FUNGAL  CUL/STN 

8 

0.000 

FUNGAL  CULT 

3388 

0.190 

FUNGAL  CULTURE 

4921 

0.276 

FUNGAL  DERM 

221 

0.012 

FUNGAL  MI SC 

1310 

0.073 

FUNGAL  SMEAR 

343 

0.019 

FUNGAL-BLOOD 

3 

0.000 

FUNGAL-BNH 

45 

0.003 

FUNGI  FILAMENTO 

22 

0.001 

FUNGI  YEASTLIKE 

121 

0.007 

FUNGUS  ID 

309 

0.017 
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Laboratory  test  order  classification 


std  test  ord 

tstorder 

COUNT 

PERCENT 

FUNGUS,  OTHER 

FUNGUS  MICRO 

31 

0.002 

KOH 

3220 

0.180 

KOH  /  WET  PREP 

10 

0.001 

KOH  PREP 

1946 

0.109 

KOH/NS 

1 

0.000 

KOH/WET  PREP 

2283 

0.128 

MYCO  CULT 

2 

0.000 

MYCOL  C&SM 

361 

0.020 

MYCOLOGY 

129 

0.007 

MYCOLOGY  CUL 

93 

0.005 

MYCOLOGY  CULT 

1399 

0.078 

MYCOLOGY  SMEAR 

32 

0.002 

NS  PREP 

24 

0.001 

R/O  DERMATOPHYT 

1 

0.000 

SALINE  PREP, SKIN 

10 

0.001 

WET  PREP, NAIL 

143 

0.008 

WET  PREP, SCALP 

30 

0.002 

WET  PREP, SKIN 

217 

0.012 

FUNGUS,  OTHER 

30048 

1.683 

FUNGUS,  RESP 

C  FUNGAL, RESP 

32 

0.002 

FUNG  ID, RESP 

4 

0.000 

FUNGAL  CULT, RESP 

671 

0.038 

FUNGAL  CULTURE, RESP 

562 

0.031 

FUNGAL  MI SC, RESP 

458 

0.026 

FUNGAL  SMEAR, RESP 

12 

0.001 

FUNGAL, RESP 

285 

0.016 

FUNGAL- BNH, RESP 

1 

0.000 

FUNGI  FILAMENTO, RESP 

21 

0.001 

FUNGI  YEASTLIKE, RESP 

11 

0.001 

FUNGUS  ID, RESP 

9 

0.001 

FUNGUS  MICRO, RESP 

2 

0.000 

KOH  PREP, RESP 

5 

0.000 

MYCOL  C&SM, RESP 

8 

0.000 

MYCOLOGY  CULT, RESP 

44 

0.002 

MYCOLOGY, RESP 

2 

0.000 

FUNGUS,  RESP 

2127 

0.119 

GC  CULT/ SMEAR/ P 

GC  (DNA  PROBE) 

17 

0.001 

GC  CULT 

64194 

3.596 

GC  CULTURE 

14013 

0.785 

GC  CX 

6 

0.000 

GC  PROBE 

94 

0.005 

GC  SMEAR 

10 

0.001 

GC/CHL  PROBE 

40135 

2.249 

GC/CHLAM 

2791 

0.156 

GC/CHLAM  PROBE 

9290 

0.520 

GC/CHLAMYDIA  PR 

22380 

1 .254 

GC/CT  PANEL 

102 

0.006 

GCSMEAR 

3 

0.000 

GISP 

263 

0.015 
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Laboratory  test 
std_test_ord 
GC  CULT/ SMEAR/ P 

GC  CULT/ SMEAR/ P 
GENITAL  CULTURE 


GENITAL  CULTURE 
GIARDIA/ CRYPTO 


GIARDIA/ CRYPTO 
GRAM  STAIN 


GRAM  STAIN 
GROUP  A  STREP 


order  classification 


tstorder 

COUNT 

PERCENT 

N  GONORRHOEAE 

485 

0. 

.  027 

R/O  GC 

114 

0. 

.  006 

153897 

8. 

.  622 

AEROBIC  CULT, GENITAL 

24 

0. 

.  001 

AEROBIC  CULTURE, GENITAL 

1 

0. 

.  000 

ANAER  CULT, GENITAL 

5 

0. 

.  000 

CULT  &  SENS, GENITAL 

2 

0. 

.  000 

CULTURE, GENITAL 

2 

0. 

.  000 

GENITAL  CULTURE 

27316 

1 . 

.  530 

MISC  CUL, GENITAL 

6 

0. 

.  000 

MISCELLANEOUS, VAG  CS 

35 

0. 

.  002 

OTHER  CULTURE, GENITAL 

3 

0. 

.  000 

VAG  CULT 

67 

0. 

.  004 

VAGINAL  CULTURE 

181 

0. 

.  010 

27642 

1 . 

.  549 

CRYPTOSPORI  STN 

15 

0. 

.  001 

CRYPTOSPORIDIUM 

53 

0. 

.  003 

CRYPTOSPORIDUM= 

1 

0. 

.  000 

CRYSPOR 

3 

0. 

.  000 

GARDIA/CRYP  DFA 

12 

0. 

.  001 

GIA  &  CRYPTO 

14 

0. 

.  001 

GIARD  LAMB  AG 

62 

0. 

.  003 

GIARDIA  AG 

40 

0. 

.  002 

GIARDIA, DFA 

65 

0. 

.  004 

GIARDIA/CRYP  SC 

849 

0. 

.  048 

1114 

0. 

.062 

GM  STAIN 

1 

0. 

.  000 

GRAM  STAIN 

10719 

0. 

.  601 

GRAM  STAIN  SPEC 

8 

0. 

.  000 

GRAM  STN 

23 

0. 

.  001 

GS  &  CULTURE 

12 

0. 

.  001 

GS  AND  CULTURE 

41 

0. 

.  002 

10804 

0. 

.  605 

GRP  A  STREP  CUL 

3866 

0. 

.217 

R/O  STREP 

4549 

0. 

.255 

RAP  STREP 

658 

0. 

.  037 

RAP  STREP-KAHC 

857 

0. 

.  048 

RAPID  STREP 

28167 

1 . 

.  578 

RAPID  STREP  A 

5858 

0. 

.  328 

RAPSTP&C 

397 

0. 

.  022 

RPDSTP-C 

191 

0. 

.  Oil 

RS  CONFIRM 

6325 

0. 

.  354 

STREP  A  PANEL 

1230 

0. 

.  069 

TC  STREP 

6088 

0. 

.  341 

58186 

3. 

.260 

GROUP  A  STREP 
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Laboratory  test 
std_test_ord 
GROUP  B  STREP 


GROUP  B  STREP 
H  DUCREYI  CULT 
H  PYLORI  CULT 


H  PYLORI  CULT 


order  classification 


tstorder 

COUNT 

PERCENT 

B  STREP  A 

16 

0. 

.  001 

BETA  STREP  CUL 

140 

0. 

.  008 

BETASTRP 

5 

0. 

.  000 

C  B  STREP 

2207 

0. 

.  124 

C&S  GBS  OB/NEO 

4850 

0. 

.272 

CERVICAL  SCR-B 

975 

0. 

.  055 

CUL  GBS  VAG 

1267 

0. 

.  071 

GBBS 

282 

0. 

.  016 

GBS 

1143 

0. 

.064 

GBS  CULTURE 

613 

0. 

.  034 

GBS  OB/GYN 

1676 

0. 

.  094 

GEN, ROBS 

475 

0. 

.  027 

GEN/STREP  SCRN 

408 

0. 

.  023 

GP  B  BETA  STREP 

2500 

0. 

.  140 

GP  B  ST 

1861 

0. 

.  104 

GP/B/CUL 

1 

0. 

.  000 

GPB  CUL 

1 

0. 

.  000 

GROUP  B  CULTURE 

196 

0. 

.  Oil 

GROUP  B  SCR 

718 

0. 

.  040 

GROUP  B  SCREEN 

1071 

0. 

.  060 

GROUP  B  STREP 

572 

0. 

.  032 

GROUP  B  STREP. 

28 

0. 

.  002 

GRP  B  PEN  ALL 

26 

0. 

.  001 

GRP  B  STREP 

2464 

0. 

.  138 

GRP  B  STREP  CUL 

541 

0. 

.  030 

OB  PANEL  X3 

348 

0. 

.  019 

OB/GYN  STREP  B 

1992 

0. 

.  112 

R/O  BETA  STREP 

5771 

0. 

.  323 

R/O  GBS 

4250 

0. 

.238 

R/O  GP  B  STREP 

3303 

0. 

.  185 

R/O  GRP  B  STREP 

2373 

0. 

.  133 

RECTAL  GROUP  B 

340 

0. 

.  019 

S  AGALACT IAE  ID 

1964 

0. 

.  110 

STREP  AGAL  CULT 

141 

0. 

.  008 

STREP  B-HEMOLY 

494 

0. 

.  028 

STREP  SCREEN  NOT  THR 

1207 

0. 

.  068 

VAG/REC  GBS 

802 

0. 

.  045 

47021 

2  . 

.  634 

H  DUCREYI  CULT 

8 

0. 

.  000 

CLO 

1 

0. 

.  000 

CLO  TEST 

668 

0. 

.  037 

H  PYLORI  CULTUR 

204 

0. 

.  011 

H . PYLORI 

15 

0. 

.  001 

HELICOBACTER  P 

976 

0. 

.  055 

1864 

0. 

.  104 

HER  DFA 

1 

0. 

.  000 

HERPES  CULT 

4444 

0. 

.249 

HERPES  CULTURE 

353 

0. 

.  020 

HERPES 


146 


Laboratory  test  order  classification 


std  test  ord 

tstorder 

COUNT 

PERCENT 

HERPES 

HSV  CULT/AG 

1 

0.000 

HSV  CULTURE 

142 

0.008 

HSV  DFA 

160 

0.009 

OB  HERPES  CULT 

4 

0.000 

HERPES 

5105 

0.286 

INFLUENZA 

FLU  A  PANEL 

608 

0.034 

INFLU  SURVEY 

337 

0.019 

INFLUENZA 

945 

0.053 

LEGIONELLA  CUL 

LDFA 

3 

0.000 

LEGIONELLA  CUL 

31 

0.002 

LEGIONELLA  CUL 

34 

0.002 

LEISHMANIASIS 

LEISHMANIASIS 

151 

0.008 

MICROORG  ID 

MICROORG  ID 

33 

0.002 

MI SC  CULTURE 

BIO  C&S 

1 

0.000 

BIOPSY  CULT 

2 

0.000 

BODYSITE  CULT 

1 

0.000 

BR  BRUSH  C&S 

8 

0.000 

BR  LAVAGE  C&S 

48 

0.003 

BRONCH  CULTURE 

34 

0.002 

BSGB  CX 

7 

0.000 

C  IUD 

12 

0.001 

C&S  GASTRIC  ASP 

4 

0.000 

C&S  MOUTH 

22 

0.001 

C&S  SKIN 

856 

0.048 

CF  CULTURE 

80 

0.004 

CULT  &  SENS 

66 

0.004 

CULT, MISC 

11 

0.001 

CULT, OTHER 

9 

0.001 

CULTURE 

455 

0.025 

CULTURE,  OTHER 

14 

0.001 

CX,  SURGICAL 

15 

0.001 

DRAIN/CYST  CX 

40 

0.002 

IUD 

1 

0.000 

JOINT  C&S 

8 

0.000 

LEPTO  CULTURE 

2 

0.000 

MED  DEVICE  CX 

1 

0.000 

MISC  CS 

27 

0.002 

MISC  CUL 

118 

0.007 

MISC  CULT 

8 

0.000 

MISC  CULTURE 

648 

0.036 

MISC  CX 

266 

0.015 

MISC.  C 

1 

0.000 

MISCELLAN.  CX 

20 

0.001 

MUMPS  CULTURE 

1 

0.000 

OTHER  CULTURE 

391 

0 . 022 
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Laboratory  test  order  classification 


std  test  ord 

tstorder 

COUNT 

PERCENT 

MI  SC  CULTURE 

RECT  CX 

2 

0.000 

SM  CULT 

8 

0.000 

SURF  C&S 

6 

0.000 

MI  SC  CULTURE 

3193 

0.179 

MRSA 

MRSA 

43 

0.002 

MRSA  CULTURE 

13 

0.001 

R/O  MRSA 

1117 

0.063 

STAPH  ID 

13 

0.001 

MRSA 

1186 

0.066 

MYCOPLASMA  CULT 

M  PNEUMO  CULT 

1 

0.000 

MYCOPLASA  ID 

2 

0.000 

MYCOPLASMA  CULT 

5 

0.000 

MYCOPLASMA  CULT 

8 

0.000 

NASAL  CULT 

AER  CULT, NASAL 

1 

0.000 

AEROBIC  CULT, NASAL 

721 

0.040 

AEROBIC  CULT, THROAT, NASAL 

16 

0.001 

AEROBIC  CULT, THROAT, NASAL, NASAL 

7 

0.000 

AEROBIC  CULTURE, NASAL 

28 

0.002 

ANAER  CULT, NASAL 

24 

0.001 

ANAEROBE-APATH, NASAL 

10 

0.001 

ANAEROBIC  PANEL, NASAL 

70 

0.004 

C  NASAL/SINUS 

132 

0.007 

CULTURE, NASAL 

11 

0.001 

CULTURE , THROAT , NASAL 

2 

0.000 

MISC  CULTURE, NASAL 

7 

0.000 

MISC  CX, NASAL 

24 

0.001 

MISC.  C, NASAL 

1 

0.000 

N-P  CULTURE 

1 

0.000 

NASAL 

12 

0.001 

NASAL  CULT 

1562 

0.088 

NASAL  CULTURE 

1173 

0.066 

NASAL  CX 

7 

0.000 

NASO-PHAR  CULT 

4 

0.000 

NOSE  CULTURE 

6 

0.000 

NP 

14 

0.001 

NP  CULTURE 

45 

0.003 

OTHER  CULTURE, NASAL 

80 

0.004 

NASAL  CULT 

3958 

0 . 222 

0  &  P 

DIRECT  O&P  EXAM 

1 

0.000 

INTESTINAL  PARA 

219 

0.012 

O&P 

12 

0.001 

O&P 

980 

0.055 

O&P  CONCENTRATE 

48 

0.003 

O&P  EXAM 

2370 

0.133 

O&P  EXM 

181 

0.010 

148 


Laboratory  test  order  classification 


std  test  ord 

tstorder 

COUNT 

PERCENT 

0  &  P 

O&P  ID 

1858 

0.104 

O&P  MACRO 

5 

0.000 

O&P  MICR 

33 

0.002 

O&P  PANEL 

684 

0.038 

O&P/TRI CHROME 

3 

0.000 

O+P  EXAM 

676 

0.038 

OOCYST  EXAM 

8 

0.000 

OP  DIRECT 

120 

0.007 

OP  EXAM 

158 

0.009 

OP  MACRO 

4 

0.000 

OP  TRICHROME 

269 

0.015 

OVA  &  PARA 

412 

0.023 

OVA  &  PARASITE 

528 

0.030 

OVA  &  PARASITES 

2549 

0.143 

OVA & PAR 

345 

0.019 

PARASITE  EXAM 

13 

0.001 

PARASITE  TEST 

511 

0.029 

SP  O&P 

10 

0.001 

SPUTUM  PARASITE 

2 

0.000 

TRICHROME  STAIN 

14 

0.001 

0  &  P 

12013 

0.673 

ORGANISM  ID 

ECTOPARASITE 

5 

0.000 

ORGANISM  ID 

116 

0.006 

VECTOR  ID 

4 

0.000 

ORGANISM  ID 

125 

0.007 

OTHER 

BACT  ANTIGENS 

77 

0.004 

BF: CRYSTAL  EXAM 

1 

0.000 

CLINTEST 

19 

0.001 

FRUCTOSE 

14 

0.001 

GROSS  BLOOD 

3 

0.000 

GROSS  BLOOD. 

4 

0.000 

LATEX  PANEL 

31 

0.002 

MACRO 

10 

0.001 

MISC  STAIN 

1 

0.000 

MISCELLANEOUS 

383 

0.021 

PH, STOOL 

8 

0.000 

PRES  OF  SPERM 

147 

0.008 

RDSP 

62 

0.003 

RECTAL 

7 

0.000 

REFERRED  ID/SEN 

246 

0.014 

SEL  BACT  AGENT 

40 

0.002 

SEMEN  PROFILE 

224 

0.013 

STOOL  PH 

120 

0.007 

STOOL/PH 

3 

0.000 

STREP  OTHER 

10 

0.001 

WELLCOGEN  TEST 

32 

0.002 

1442  0.081 


OTHER 
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Laboratory  test 
std_test_ord 
OTHER  GI 


OTHER  GI 
OTHER  STD 


OTHER  STD 
PAP  SMEAR 

PAP  SMEAR 
P INWORM 


P INWORM 
RECTAL  CULT 

RECTAL  CULT 


order  classification 


tstorder 

COUNT 

PERCENT 

FAT, MICROSCOPIC 

91 

0. 

.  005 

FEC  MACRO  EXAM 

15 

0. 

.  001 

FECAL  EXAM 

42 

0. 

.  002 

FECAL  FAT 

51 

0. 

.  003 

FECAL  FAT , QUAL 

126 

0. 

.  007 

GASTRIC 

4 

0. 

.  000 

MACRO  EXAM  ST 00 

13 

0. 

.  001 

OCC  BLD-GASTRIC 

14 

0. 

.  001 

QUAL  FECAL 

39 

0. 

.  002 

QUAL  FECAL  FAT 

17 

0. 

.  001 

STOOL 

16 

0. 

.  001 

STOOL  FECAL  FAT 

35 

0. 

.  002 

STOOL  GR  EXAM 

423 

0. 

.  024 

886 

0. 

.  050 

C&S  GENITAL-TM 

10574 

0. 

.  592 

DARKFLD 

1 

0. 

.  000 

HPV 

5 

0. 

.  000 

HPV  PROFILE 

1 

0. 

.  000 

INP  CHLAMYDIA 

15 

0. 

.  001 

SALINE  MOUNT 

8 

0. 

.  000 

SALINE  PREP 

1693 

0. 

.  095 

UREAPLASMA  ID 

1 

0. 

.  000 

URETHRAL  C&S 

7 

0. 

.  000 

URETHRAL  CULTUR 

3 

0. 

.  000 

W-PREP 

4 

0. 

.  000 

WET  PREP 

47913 

2  . 

.  684 

WET  PREP  NS 

6 

0. 

.  000 

WET  PREP  SALINE 

12 

0. 

.  001 

60243 

3. 

.  375 

PAP 

436 

0. 

.  024 

PAP (TRICARE) 

397 

0. 

.  022 

833 

0. 

.  047 

MICRO  PIN  WORM 

97 

0. 

.  005 

PIN  WORM  EXAM 

46 

0. 

.  003 

P INWORM 

439 

0. 

.  025 

P INWORM  EXAM 

34 

0. 

.  002 

P INWORM  PREP 

177 

0. 

.  010 

WORM  ID 

16 

0. 

.  001 

809 

0. 

.  045 

RECTAL  CULT 

9 

0. 

.  001 

RECTAL  CULTURE 

5 

0. 

.  000 

14 

0. 

.  001 

CULTURE  RESPIRA 

28 

0  . 

.  002 

RESP  CULT 
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Laboratory  test  order  classification 


std  test  ord 

tstorder 

COUNT 

PERCENT 

RESP  CULT 

LOW  RESP  CULT 

48 

0.003 

LOWER  RESP  CULT 

91 

0.005 

RESP  CULT 

4735 

0.265 

RESP  PANEL 

368 

0.021 

RESP.  CULTURE 

68 

0.004 

UPPER  RESP  CUL 

65 

0.004 

UPPER  RESP  CULT 

1 

0.000 

RESP  CULT 

5404 

0.303 

RESP  CULTURE 

AEROBIC  CULT, RESP 

56 

0.003 

CULT  &  SENS, RESP 

12 

0.001 

RESP  CULTURE 

68 

0.004 

ROTAVIRUS 

ROTAVIRUS 

35 

0.002 

RSV 

RSV 

470 

0.026 

RSV  SCREEN 

212 

0.012 

RSV  TEST 

194 

0.011 

RSV 

876 

0.049 

SKIN  CULT 

AEROBIC  CULT, NASAL, SKIN 

1 

0.000 

AEROBIC  CULTURE, NASAL, SKIN 

1 

0.000 

SKIN  C&S 

71 

0.004 

SKIN  CULTURE 

122 

0.007 

SKIN  CX 

35 

0.002 

SKIN  CULT 

230 

0.013 

SPUTUM  CULT 

AEROBIC  CULT, SPUTUM 

22 

0.001 

AEROBIC  CULTURE, SPUTUM 

14 

0.001 

ANAER  CULT, SPUTUM 

1 

0.000 

ANAEROBIC  PANEL, SPUTUM 

7 

0.000 

C  SPUTUM 

614 

0.034 

C&S  SPUTUM 

587 

0.033 

CF  CULTURE, SPUTUM 

142 

0.008 

SP  C&S 

1 

0.000 

SPU  CULT 

71 

0.004 

SPUT  C&S/ SMEAR 

94 

0.005 

SPUTUM  C 

1 

0.000 

SPUTUM  C&S 

184 

0.010 

SPUTUM  C&S/ GRAM 

175 

0.010 

SPUTUM  CU 

24 

0.001 

SPUTUM  CULT 

519 

0.029 

SPUTUM  CULTURE 

760 

0.043 

SPUTUM  CX 

19 

0.001 

SPUTUM  PANEL 

80 

0.004 

S  PUTUMCX 

22 

0.001 

3337  0.187 


SPUTUM  CULT 
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Laboratory  test  order  classification 


std  test  ord 

tstorder 

COUNT 

PERCENT 

SSC 

CAMPY  CULTURE 

1 

0.000 

CAMPY  ID 

8 

0.000 

CAMPYLOBACTER 

133 

0.007 

S/S  SCREEN 

32 

0.002 

SALMONELLA  GRP 

1 

0.000 

SHIGELLA  SERO 

5 

0.000 

STOOL, SALM/SHIG 

20 

0.001 

STOOL : SSC 

13 

0.001 

STOOL : SSCE 

125 

0.007 

STOOL :SSE 

6 

0.000 

SSC 

344 

0.019 

STOOL  CULT 

AEROBIC  CULT, STOOL 

156 

0.009 

C  STOOL 

2 

0.000 

CULTURE, STOOL 

1 

0.000 

STOOL  CUL 

1 

0.000 

STOOL  CULT 

26376 

1.478 

STOOL  CULTURE 

824 

0.046 

STOOL  CX 

563 

0.032 

STOOL  CULT 

27923 

1.564 

THR  CULT 

AEROBIC  CULT, THROAT 

54 

0.003 

AEROBIC  CULTURE, THROAT 

1 

0.000 

ANAER  CULT, THROAT 

32 

0.002 

ANAEROBIC  CULT, THROAT 

2 

0.000 

CULTURE, THROAT 

3 

0.000 

MISC  CULTURE, THROAT 

9 

0.001 

MISC  CX, THROAT 

4 

0.000 

OTHER  CULTURE, THROAT 

1 

0.000 

PROJECT  GARGLE 

312 

0.017 

TC 

3811 

0.214 

TC  OTHER 

39 

0.002 

TC  SCREEN 

1111 

0.062 

THR  CULT 

413984 

23.193 

THROAT 

3187 

0.179 

THROAT  CULTURE 

29830 

1.671 

THR  CULT 

452380 

25.344 

TISSUE  CULT 

BIOPSY 

23 

0.001 

C  TISSUE 

57 

0.003 

C&S  TISSUE/BONE 

42 

0.002 

TISSUE  BK  CX 

1 

0.000 

TISSUE  C&S 

40 

0.002 

TISSUE  CULT 

113 

0.006 

TISSUE  CULTURE 

636 

0.036 

TISSUE  CULT 

912 

0.051 

UA  CULT 

AEROBIC  CULT, URINE 

5 

0.000 

ANAER  CULT, URINE 

1 

0.000 
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std  test  ord 

tstorder 

COUNT 

PERCENT 

UA  CULT 

LEPTO  CULTURE, URINE 

10 

0.001 

OB  UA 

190 

0.011 

UA  CULT 

505027 

28.293 

UA  MICRO  SCREEN 

2804 

0.157 

UA  PEDS 

1223 

0.069 

UC 

511 

0.029 

UR  CULT 

7529 

0.422 

UR  CULT  LOW 

1 

0.000 

UR  CULT, LOW 

1 

0.000 

URINE  CULT-BNH 

8438 

0.473 

URINE  CULTURE 

1616 

0.091 

URINE  CX 

3284 

0.184 

UA  CULT 

530640 

29.728 

VARICELLA 

V  ZOSTER  CULT 

2 

0.000 

VARICEL  CULT 

105 

0.006 

VZ-DFA 

2 

0.000 

VZV  CULTURE 

12 

0.001 

VZV  DFA 

12 

0.001 

VARICELLA 

133 

0.007 

VIBRIO 

VIB-AER 

19 

0.001 

VIB/YER/E0157 

206 

0.012 

VIBRIO 

2 

0.000 

VIBRIO  SP  IDENT 

1 

0.000 

VIBRIO 

228 

0.013 

VIRAL  CULT 

ENTEROVIRUS  CUL 

8 

0.000 

RESP  VIRUS  CULT 

438 

0.025 

VIRAL  CULT 

1959 

0.110 

VIRAL  CULTURE 

349 

0.020 

VIRAL  CX 

6 

0.000 

VIRUS  ID 

251 

0.014 

VIRAL  CULT 

3011 

0.169 

VIRAL  PANEL 

VIRAL  PNL 

19 

0.001 

VRE 

R/O  VRE 

4 

0.000 

WND  CULT 

ABSCESS  CULTURE 

287 

0.016 

AER  WOUND/GRAM 

93 

0.005 

AEROBIC  CULT, WND 

652 

0.037 

ANA  WOUND/GRAM 

3 

0.000 

ANAER  CULT, WND 

70 

0.004 

ANAEROBE-APATH, WND 

5 

0.000 

ANAEROBIC  PANEL, WND 

38 

0.002 

C&S  WOUND 

2 

0.000 

CULTURE, WOUND 

294 

0.016 

LESION  CX 

11 

0.001 
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std  test  ord 

tstorder 

COUNT 

PERCENT 

WND  CULT 

WAR  WOUND 

112 

0.006 

WND  CULT 

30696 

1.720 

WOUND  ANA-BNH 

276 

0.015 

WOUND  CULTURE 

3817 

0.214 

WOUND  CX 

148 

0.008 

WND  CULT 

36504 

2.045 

YERSINIA  CULT 

YER.CUL 

32 

0.002 

YER/VIB 

35 

0.002 

YERSIN 

8 

0.000 

YERSINIA  CULT 

3 

0.000 

YERSINIA  CULTUR 

88 

0.005 

YERSINIA  SP  ID 

1 

0.000 

YERSINIA  CULT 

167 

0.009 

1784959 

100.000 
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